Posts

Unit testing ILogger with NSubstitute and time information

There are several problems when testing a method like protected override async Task ExecuteAsync(CancellationToken stoppingToken) { logger.LogInformation("QueueProcessorService started at: {time:g}", timeProvider.GetUtcNow()); while (!stoppingToken.IsCancellationRequested) { await processor.TryProcessingAsync(stoppingToken).ConfigureAwait(false); await Task.Delay(Constants.QUEUE_POLLING_INTERVAL).ConfigureAwait(false); } logger.LogInformation("QueueProcessorService stopped at: {time:g}", timeProvider.GetUtcNow()); } and more specifically the logger.LogInformation calls. For one thing, doing a simple await Task.Delay(Constants.QUEUE_POLLING_INTERVAL).ConfigureAwait(false); will actually slow down the test for however long the specified interval is. This is unacceptable for any significant value of QUEUE_POLLING_INTERVAL (the production code might only want to check the queue once...

A generic database class

This is a class I use frequently for running SQL commands / queries: public   class   Database  :  IDatabase {      public   Database ( DatabaseConfiguration   config )     {          this . config   =   config ;     }      public   object [][]  RunQuery ( string   query ,  params   object []  args )     {          using  ( var   con   =   Connect ())          using  ( var   cmd   =   Prepare ( con ,  query ,  args ))              return   cmd . ExecuteReader () . Pipe ( ReadData );     }      public   void   RunNonQuery ( string ...

The early bird catches the worm

Here are three reasons to do things later rather than earlier: You will know more. You will usually be richer and thus can buy some help (at least for part of the task). You might be dead and therefore no longer care about the task :)

On Dependency Injection

I just read an article about eliminating dependencies whose basic thesis is that instead of having a class that depends on something that can give you a value, just depend on that value directly. His example (which you can read more fully in the article) is that instead of having public InvoiceGenerator(IConfigurationReader configurationReader) { _configurationReader = configurationReader; } and later on calling var watermarkText = _configurationReader.Get<string>("invoiceWatermarkText"); if (!String.IsNullOrEmpty(watermarkText)) to get the text we need, just request the text directly: public Invoice GenerateInvoice(string watermarkText) { if (!String.IsNullOrEmpty(watermarkText)) There are two obvious problems with this, which you will hit extremely quickly in real code. One of them, pointed out by a comment by LeszekP, is that most of the time you will require more than a single value from an interface. ...

My views on blockchain / bitcoin

Going from abstract to concrete: Private currency: good. I would love to have lots of them, just as I love both gold and silver. Blockchain: not so good, more of a solution in search of a problem. The only advantage it has compared to an append-only database is decentralization… and I don't see a need for that. I would much rather have many private currencies. Also, in practice most coins are actually centralized, or at least a very small oligarchy. Bitcoin: nah. Really bad. Incredibly slow to initialize, slow transactions, easy to control by a sustained state-level effort, very low cap on number of transactions per second, plenty of bugs, hard-forks which means the code is actually controlled by a small group (which in turn means that I can't trust that the currency itself is decentralized)… nah. In conclusion, I like the initiative, and I'm 100% behind the idea that we need to separate states and money, but we're nowhere near yet.

The problem with null

The problem with null is that it pretends to be an object of a given type, without actually having that type. For example (C# code - ignore the uselessness of the GetName method): string GetName(Customer customer) { // I got a customer object, I can access the Name property return customer.Name; } var x = func(null); // <-- not a real Customer object so we have a run-time error The proper way to solve this is by using the Option (aka Maybe) monad; for an example using the Functional.Maybe NuGet package: string GetName(Maybe<Customer> customer) { // I don't actually have a Customer object, I have a Maybe<Customer> // I need to treat it carefully return customer.Select(it => it.Name).OrElse("Missing"); } var x = func(null); // compiler error, because Maybe<> doesn't allow null as a value var x = func(Maybe<Customer>.Nothing); // the customer is missing but the call will not crash Using the Opt...

FanFiction

FanFiction Phrase from a book I'm reading: I was bitten by a radioactive Jedi as a child. The Havoc side of the Force

Fan-fiction recommendations

I posted this to HN and I realized it might be useful for other people. If at least one person who wasn't aware of fan-fiction discovers it as a result, this has served its purpose. For Buffy fans, TTH has a huge number of stories, both in-universe and crossovers with other worlds. Speakertocustomers , Becuzitswrong , Cordyfan , DianeCastle and Hotpoint are a few of my favorite authors. Hotpoint's crossover between X-COM and SG-1 is absolutely amazing. For other worlds, FF , SB , SV and AO3 are the most well-known sites for fanfiction. One of my favorite stories is Taylor Varga . What I normally do is I download the stories in the .mobi format - TTH and AO3 have that feature in the site, and for the others you can use the FanFicFare plugin for Calibre - and then upload them to my Kindle. This allows me to read in bed, which is bad for my sleep but I like it :)

Don't compare floats

Floating-point numbers are tricky; one of the first things a programmer needs to remember when working with them is to never check floats for equality. (I believe ReSharper warns you if you do that; I don't know if plain Visual Studio does because I never use it without ReSharper.) If precise representations of decimal numbers is needed, like when manipulating currencies, use decimals instead (if using C#); if a similar primitive type does not exist in your language, write a separate package / module / library to "fake it" by using integer values and scale them down by two or four digits, depending on your needs. (For example, the number 123456 can represent either 1,234.56 or 12.3456 , depending on your application.) Here's a simple code example to show the difference between float s and decimal s in C#: float f1 = 0.1f; float f2 = f1 * 10.0f; float f3 = 0.0f; for (var i = 1; i <= 10; i++)     f3 += f1; Console.WriteLine(f2); Console.WriteLine(f3); Consol...

Evaluating an expression

Leaving a note to myself - a simple algorithm for evaluating expressions, no unary operators or parentheses. phase 1 (tokenizer) '+' => op1, add '-' => op1, sub '*' => op2, mul '/' => op2, div \d+ => number, value phase 2 (evaluator) tuple = first (number, op2, number) while tuple: replace tuple with result of op2 tuple = first (number, op1, number) while tuple: replace tuple with result of op1 there should be a single item left, a number; return its value

Returning an IEnumerable from a database

This is probably rather obvious, but... when returning an `IEnumerable` from a database, like the records from a table, don't do this: using (var db = GetDatabase()) { return db.GetTable("Table1"); } because it will throw an exception when trying to enumerate those records, since the database connection has already been disposed. Don't do this either: using (var db = GetDatabase()) { return db.GetTable("Table1").ToList(); } because it will retrieve all records from the table, even if you only need a small subset. A slightly better way is to do this: using (var db = GetDatabase()) { foreach (var item in db.GetTable("Table1")) yield return item; } This way, the `Dispose` method doesn't get called until the enumeration is over and if you only `Take()` a limited number of records from the result, it won't load the whole table. On the other hand, if you only add `Where()` clauses to the result, it will still enumerate every...

Aligning text

I needed to write some code in a console app to align what a user was saying, in case it was longer than a line (80 characters): marcel: blah blah blah a lot of text that doesn't fit in 80 characters more blah blah blah long_username: Contrary to popular belief, Lorem Ipsum is not simply random text . It has roots in a piece of classical Latin literature from 45 B C, making it over 2000 years old. Richard McClintock, a Latin pro fessor at Hampden-Sydney College in Virginia, looked up one of th I wrote the code in two ways: an imperative, mutating style and a functional (recursive) style. I find the second one to be more elegant but since the entire project is something done as a hobby I don't much care about speed; your mileage might vary. I'm also quite certain the first method can be improved but… again, I don't need to do that so it doesn't get done. As usual, use at your own risk, I don't care about c...

Math is a game

Math is a game, an arbitrary set of symbols and rules. The weird part, the part that always surprises me, is that it's relevant to the real world. Here's an example: we'll start with just two symbols, Yin and Yang. (Replace those with black and white, circle and square, X and Y… whichever two symbols you prefer.) What can we do with them? Well, the simplest thing we can do is transform one into the other: Yin → Yang; Yang → Yin We'll call this transformation "mirroring" and denote it with the letter "M". What about combining two symbols? We have a number of possibilities: A) Yin, Yang → Yin; Yang, Yin → Yin; Yin, Yin → Yin; Yang, Yang → Yin This is rather boring… no matter what we start with, we obtain an Yin symbol. Nevertheless, let's continue. B) The opposite of A: Yin, Yang → Yang; Yang, Yin → Yang; Yin, Yin → Yang; Yang, Yang → Yang. Still boring. C) Yin, Yang → Yin; Yang, Yin → Yin; Yin, Y...

Parsing INI files

I am trying to allow the end user to modify the application behavior in some limited ways; as such, I have a need of parsing .INI files of the form [Type 1] contains=abc contains=def [Type 2] contains=1234 I looked around for a class / library that would allow me to read these files but I haven't found anything useful (most of the classes I found couldn't handle duplicate keys within the same section). As such, I went ahead and implemented this myself. This algorithm is extremely specific to my usage, you might have to modify it for your needs. The IniTuple represents each line as a (section, key, value) tuple: public class IniTuple { public string Section { get; private set; } public string Key { get; private set; } public string Value { get; private set; } public IniTuple(string section, string key, string value) { Section = section; Key = key; Value = value; } } The IniP...

C# math is fast

I was reading an article on neural networks that mentioned the usual sigmoid activation function when the inputs are real numbers in the [0, 1) interval: 1 / (1 + e −x ) The article mentions that this is probably where the program would spend at least half of its time so I thought "why not pre-compute a bunch of values and trade memory and precision for time"? It turns out, C# math is quite fast and the gain might not be worth it. (I haven't tested it yet with a NN.) This is the code I wrote to benchmark the two options, using LinqPad 5 : void Main() { const int STEPS = 1 * 1000 * 1000; Func<double, double> activation = x => 1.0 / (1.0 + Math.Exp(-x)); var cache = Precompute(0.0, 1.0, STEPS, activation); Benchmark("Using the cache", x => cache[(int) Math.Truncate(x * STEPS)]); Benchmark("Calling the function each time", activation); } double[] Precompute(double lower, double upper, int steps, Func<dou...

A simple rules engine

I'm extracting data from some OCR'd letters and, in order to determine which type of letter I'm parsing, I'm using a method similar to this: public Letter Parse(string text) { Letter result; if (text.IndexOf("...", StringComparison.OrdinalIgnoreCase) >= 0) letter = new LetterA(); else letter = new LetterB(); //... additional processing return letter; } If the letter contains a specific text, I know it's of one type; otherwise I'll default to the other type. Unfortunately that's going to get really complicated, really fast once I start adding new letter types. I read somewhere that "you should move logic out of the code and into the data when possible"; it made sense and I never had a reason to regret it. So, let me try to do that here. First I'll add a "rules list" class that will allow me to store the ...

Crystal Reports woes

This took me an hour to figure out so I thought I'd write it down in case it helps anyone else. If you have a form that's going to display a Crystal Report and you want to zoom it by default, the "normal" way would be to do this in form_Shown: private void ReportViewer_Shown(object sender, EventArgs e) { viewer.Zoom(2); // 1 = page width, 2 = whole page, 25..400 = zoom factor } (Where viewer is the CrystalReportViewer component.) Unfortunately, it takes CR a while to compute and display the actual report; by the time that happens, the .Zoom() call has already been executed (and ignored). I have tried a number of workarounds (including launching a thread, waiting for two seconds and then calling the Zoom method - it worked but it was a horrible hack) before I discovered that CR has a "hidden" PageChanged event (it has a [Browsable(false)] attribute). Use that event by assigning a handler in the constructor: viewer.PageCha...

POST-ing to an ASP.NET MVC form with an anti-forgery token

I've had some issues trying to write an acceptance test that was registering a new user by POST-ing the required information to a MVC site and I got back these messages: The required anti-forgery cookie "__RequestVerificationToken" is not present. The required anti-forgery form field "__RequestVerificationToken" is not present. Validation of the provided anti-forgery token failed. The cookie "_RequestVerificationToken" and the form field "_RequestVerificationToken" were swapped. Since it took me a bit of fiddling with the code before I managed to make it work, I thought I'd share the solution. The site is an ASP.NET MVC version 5 and I am trying to register a new user (POST-ing to the /Account/Register URL). The main issue you will encounter is having to extract two anti-forgery tokens, one from the cookies and one from the form, and then sending both of them in the appropriate places (cookies vs form field). I have used LinqPad 5 w...

Processing a downloaded text file while it's downloading

I've had a job where I had to download some huge text files and I thought it interesting to process them while they are downloading, instead of waiting until the download was finished. (Time was very important.) Ultimately, the client changed his mind and wanted the whole thing downloaded first but I thought this was an interesting code fragment to save for later. When it comes to "do real-time stuff" I use the Rx-Main NuGet package . The interface is simple: public interface Downloader { /// <summary> /// Downloads a file from the given URL and returns it line by line. /// </summary> /// <param name="url">The URL of the file to be downloaded.</param> /// <returns>The lines from the downloaded file, as a stream.</returns> IObservable<string> Download(string url); } The only noteworthy thing about the implementation is the automatic decompression; using that property sends the appropr...

A data flow helper class

One problem I encounter when processing lists is exception handling. I prefer to write code that "chains" calls transforming the data: var results = list .Select(DoThing1) .Select(DoThing2) // ... .Select(DoThingN) .ToList(); The problem with something like this is that, if any of the calls throws an exception, processing stops for the whole list. Handling that requires that I move the "chain" to a new method and handle exceptions there: var results = list.Select(InnerMethod).ToList(); // ... private ResultN InnerMethod(Input input) { try { var r1 = DoThing1(input); var r2 = DoThing2(r1); // ... var rn = DoThingN(rn_1); return rn; } catch(Exception ex) { // do something with ex, like logging return ?? // can't throw, I want to continue processing the rest of the list } } Now I have two problems :) One is that the code just looks uglier, so maybe most p...