Posts

Showing posts with the label c#

Unit testing ILogger with NSubstitute and time information

There are several problems when testing a method like protected override async Task ExecuteAsync(CancellationToken stoppingToken) { logger.LogInformation("QueueProcessorService started at: {time:g}", timeProvider.GetUtcNow()); while (!stoppingToken.IsCancellationRequested) { await processor.TryProcessingAsync(stoppingToken).ConfigureAwait(false); await Task.Delay(Constants.QUEUE_POLLING_INTERVAL).ConfigureAwait(false); } logger.LogInformation("QueueProcessorService stopped at: {time:g}", timeProvider.GetUtcNow()); } and more specifically the logger.LogInformation calls. For one thing, doing a simple await Task.Delay(Constants.QUEUE_POLLING_INTERVAL).ConfigureAwait(false); will actually slow down the test for however long the specified interval is. This is unacceptable for any significant value of QUEUE_POLLING_INTERVAL (the production code might only want to check the queue once...

A generic database class

This is a class I use frequently for running SQL commands / queries: public   class   Database  :  IDatabase {      public   Database ( DatabaseConfiguration   config )     {          this . config   =   config ;     }      public   object [][]  RunQuery ( string   query ,  params   object []  args )     {          using  ( var   con   =   Connect ())          using  ( var   cmd   =   Prepare ( con ,  query ,  args ))              return   cmd . ExecuteReader () . Pipe ( ReadData );     }      public   void   RunNonQuery ( string ...

On Dependency Injection

I just read an article about eliminating dependencies whose basic thesis is that instead of having a class that depends on something that can give you a value, just depend on that value directly. His example (which you can read more fully in the article) is that instead of having public InvoiceGenerator(IConfigurationReader configurationReader) { _configurationReader = configurationReader; } and later on calling var watermarkText = _configurationReader.Get<string>("invoiceWatermarkText"); if (!String.IsNullOrEmpty(watermarkText)) to get the text we need, just request the text directly: public Invoice GenerateInvoice(string watermarkText) { if (!String.IsNullOrEmpty(watermarkText)) There are two obvious problems with this, which you will hit extremely quickly in real code. One of them, pointed out by a comment by LeszekP, is that most of the time you will require more than a single value from an interface. ...

The problem with null

The problem with null is that it pretends to be an object of a given type, without actually having that type. For example (C# code - ignore the uselessness of the GetName method): string GetName(Customer customer) { // I got a customer object, I can access the Name property return customer.Name; } var x = func(null); // <-- not a real Customer object so we have a run-time error The proper way to solve this is by using the Option (aka Maybe) monad; for an example using the Functional.Maybe NuGet package: string GetName(Maybe<Customer> customer) { // I don't actually have a Customer object, I have a Maybe<Customer> // I need to treat it carefully return customer.Select(it => it.Name).OrElse("Missing"); } var x = func(null); // compiler error, because Maybe<> doesn't allow null as a value var x = func(Maybe<Customer>.Nothing); // the customer is missing but the call will not crash Using the Opt...

Don't compare floats

Floating-point numbers are tricky; one of the first things a programmer needs to remember when working with them is to never check floats for equality. (I believe ReSharper warns you if you do that; I don't know if plain Visual Studio does because I never use it without ReSharper.) If precise representations of decimal numbers is needed, like when manipulating currencies, use decimals instead (if using C#); if a similar primitive type does not exist in your language, write a separate package / module / library to "fake it" by using integer values and scale them down by two or four digits, depending on your needs. (For example, the number 123456 can represent either 1,234.56 or 12.3456 , depending on your application.) Here's a simple code example to show the difference between float s and decimal s in C#: float f1 = 0.1f; float f2 = f1 * 10.0f; float f3 = 0.0f; for (var i = 1; i <= 10; i++)     f3 += f1; Console.WriteLine(f2); Console.WriteLine(f3); Consol...

Returning an IEnumerable from a database

This is probably rather obvious, but... when returning an `IEnumerable` from a database, like the records from a table, don't do this: using (var db = GetDatabase()) { return db.GetTable("Table1"); } because it will throw an exception when trying to enumerate those records, since the database connection has already been disposed. Don't do this either: using (var db = GetDatabase()) { return db.GetTable("Table1").ToList(); } because it will retrieve all records from the table, even if you only need a small subset. A slightly better way is to do this: using (var db = GetDatabase()) { foreach (var item in db.GetTable("Table1")) yield return item; } This way, the `Dispose` method doesn't get called until the enumeration is over and if you only `Take()` a limited number of records from the result, it won't load the whole table. On the other hand, if you only add `Where()` clauses to the result, it will still enumerate every...

Aligning text

I needed to write some code in a console app to align what a user was saying, in case it was longer than a line (80 characters): marcel: blah blah blah a lot of text that doesn't fit in 80 characters more blah blah blah long_username: Contrary to popular belief, Lorem Ipsum is not simply random text . It has roots in a piece of classical Latin literature from 45 B C, making it over 2000 years old. Richard McClintock, a Latin pro fessor at Hampden-Sydney College in Virginia, looked up one of th I wrote the code in two ways: an imperative, mutating style and a functional (recursive) style. I find the second one to be more elegant but since the entire project is something done as a hobby I don't much care about speed; your mileage might vary. I'm also quite certain the first method can be improved but… again, I don't need to do that so it doesn't get done. As usual, use at your own risk, I don't care about c...

Parsing INI files

I am trying to allow the end user to modify the application behavior in some limited ways; as such, I have a need of parsing .INI files of the form [Type 1] contains=abc contains=def [Type 2] contains=1234 I looked around for a class / library that would allow me to read these files but I haven't found anything useful (most of the classes I found couldn't handle duplicate keys within the same section). As such, I went ahead and implemented this myself. This algorithm is extremely specific to my usage, you might have to modify it for your needs. The IniTuple represents each line as a (section, key, value) tuple: public class IniTuple { public string Section { get; private set; } public string Key { get; private set; } public string Value { get; private set; } public IniTuple(string section, string key, string value) { Section = section; Key = key; Value = value; } } The IniP...

C# math is fast

I was reading an article on neural networks that mentioned the usual sigmoid activation function when the inputs are real numbers in the [0, 1) interval: 1 / (1 + e −x ) The article mentions that this is probably where the program would spend at least half of its time so I thought "why not pre-compute a bunch of values and trade memory and precision for time"? It turns out, C# math is quite fast and the gain might not be worth it. (I haven't tested it yet with a NN.) This is the code I wrote to benchmark the two options, using LinqPad 5 : void Main() { const int STEPS = 1 * 1000 * 1000; Func<double, double> activation = x => 1.0 / (1.0 + Math.Exp(-x)); var cache = Precompute(0.0, 1.0, STEPS, activation); Benchmark("Using the cache", x => cache[(int) Math.Truncate(x * STEPS)]); Benchmark("Calling the function each time", activation); } double[] Precompute(double lower, double upper, int steps, Func<dou...

A simple rules engine

I'm extracting data from some OCR'd letters and, in order to determine which type of letter I'm parsing, I'm using a method similar to this: public Letter Parse(string text) { Letter result; if (text.IndexOf("...", StringComparison.OrdinalIgnoreCase) >= 0) letter = new LetterA(); else letter = new LetterB(); //... additional processing return letter; } If the letter contains a specific text, I know it's of one type; otherwise I'll default to the other type. Unfortunately that's going to get really complicated, really fast once I start adding new letter types. I read somewhere that "you should move logic out of the code and into the data when possible"; it made sense and I never had a reason to regret it. So, let me try to do that here. First I'll add a "rules list" class that will allow me to store the ...

Crystal Reports woes

This took me an hour to figure out so I thought I'd write it down in case it helps anyone else. If you have a form that's going to display a Crystal Report and you want to zoom it by default, the "normal" way would be to do this in form_Shown: private void ReportViewer_Shown(object sender, EventArgs e) { viewer.Zoom(2); // 1 = page width, 2 = whole page, 25..400 = zoom factor } (Where viewer is the CrystalReportViewer component.) Unfortunately, it takes CR a while to compute and display the actual report; by the time that happens, the .Zoom() call has already been executed (and ignored). I have tried a number of workarounds (including launching a thread, waiting for two seconds and then calling the Zoom method - it worked but it was a horrible hack) before I discovered that CR has a "hidden" PageChanged event (it has a [Browsable(false)] attribute). Use that event by assigning a handler in the constructor: viewer.PageCha...

POST-ing to an ASP.NET MVC form with an anti-forgery token

I've had some issues trying to write an acceptance test that was registering a new user by POST-ing the required information to a MVC site and I got back these messages: The required anti-forgery cookie "__RequestVerificationToken" is not present. The required anti-forgery form field "__RequestVerificationToken" is not present. Validation of the provided anti-forgery token failed. The cookie "_RequestVerificationToken" and the form field "_RequestVerificationToken" were swapped. Since it took me a bit of fiddling with the code before I managed to make it work, I thought I'd share the solution. The site is an ASP.NET MVC version 5 and I am trying to register a new user (POST-ing to the /Account/Register URL). The main issue you will encounter is having to extract two anti-forgery tokens, one from the cookies and one from the form, and then sending both of them in the appropriate places (cookies vs form field). I have used LinqPad 5 w...

Processing a downloaded text file while it's downloading

I've had a job where I had to download some huge text files and I thought it interesting to process them while they are downloading, instead of waiting until the download was finished. (Time was very important.) Ultimately, the client changed his mind and wanted the whole thing downloaded first but I thought this was an interesting code fragment to save for later. When it comes to "do real-time stuff" I use the Rx-Main NuGet package . The interface is simple: public interface Downloader { /// <summary> /// Downloads a file from the given URL and returns it line by line. /// </summary> /// <param name="url">The URL of the file to be downloaded.</param> /// <returns>The lines from the downloaded file, as a stream.</returns> IObservable<string> Download(string url); } The only noteworthy thing about the implementation is the automatic decompression; using that property sends the appropr...

A data flow helper class

One problem I encounter when processing lists is exception handling. I prefer to write code that "chains" calls transforming the data: var results = list .Select(DoThing1) .Select(DoThing2) // ... .Select(DoThingN) .ToList(); The problem with something like this is that, if any of the calls throws an exception, processing stops for the whole list. Handling that requires that I move the "chain" to a new method and handle exceptions there: var results = list.Select(InnerMethod).ToList(); // ... private ResultN InnerMethod(Input input) { try { var r1 = DoThing1(input); var r2 = DoThing2(r1); // ... var rn = DoThingN(rn_1); return rn; } catch(Exception ex) { // do something with ex, like logging return ?? // can't throw, I want to continue processing the rest of the list } } Now I have two problems :) One is that the code just looks uglier, so maybe most p...

Retry algorithm

Retry with exponential back-off Update on July 24, 2016: I just discovered Polly , which does this and more a lot better. I think this is an useful class so I'm just going to leave it here. (I'm annoyed by the duplication between Retry and RetryAsync but I haven't been able to remove it.) public interface RetryPolicy { T Retry<T>(Func<T> func); void Retry(Action action); Task<T> RetryAsync<T>(Func<Task<T>> func); Task RetryAsync(Func<Task> action); } public class RetryPolicyWithExponentialDelay : RetryPolicy { // ReSharper disable once InconsistentNaming public Func<double> GetRandom = () => RND.NextDouble(); // ReSharper disable once InconsistentNaming public Action<int> Sleep = timeout => Thread.Sleep(timeout); public RetryPolicyWithExponentialDelay(int maxCount, TimeSpan initialDelay, TimeSpan maxDelay) { this.maxCount = maxCount; this.i...

Stupid code fragments, part two

Probabilities are hard. As an example, there's a known puzzle: a family has two children; if one of them is a girl, what is the probability that the other one is also a girl? The answer, un-intuitively, is not 1/2 but 1/3. There are various explanations but – as with the Monty Python puzzle years ago – I wanted to write code to check it out, so I wrote the following using LinqPad: void Main() { var rnd = new Random(); // Generate a random set of families with two children; true means girl, false means boy var all = Enumerable.Range(1, 10000).Select(_ => new Pair(rnd.Next(2) == 0, rnd.Next(2) == 0)).ToList(); // Extract only the families with at least one girl var oneGirl = all.Where(it => it.First || it.Second).ToList(); // Out of those families, how many have two girls? The result should be 1/3rd var otherGirl = oneGirl.Where(it => it.First && it.Second).ToList(); Console.Wri...

Stupid code fragments, part one

I just discovered a surprisingly simple (and obvious in hindsight) algorithm for calculating the week index of a given date. For example, April 15th is in the 3rd week (or the 3rd Tuesday of the month). I was going to do the usual thing and just Google for it but then I realized that the solution is extremely simple: private static int GetWeekPosition(DateTime date) { // the position of the given date is how many times I can subtract 7 days (go back one week) and still be in the same month // in other words, it's the integer part of (day / 7) return date.Day / 7; } (I am returning a base-zero result, but you can of course add 1 if you need it.) I realize this is not the answer to the Universe or anything but I thought it's interesting.

Avoid boolean parameters

Here's an example of a method in a sim. This started out as a simple method: all days are the same in our simulation, so the method just prints out some activities: void SimulateDay() { Console.WriteLine("Wake up."); Console.WriteLine("Eat breakfast."); Console.WriteLine("Watch TV."); Console.WriteLine("Eat lunch."); Console.WriteLine("Watch TV."); Console.WriteLine("Eat dinner."); Console.WriteLine("Go to sleep."); } Ok, so not much of a life, but this article is supposed to be about programming. :) A new requirement comes up: Sunday should be different. Well, almost; it's actually the same as the other days with a single difference: instead of watching TV, the sim goes to church on Sunday mornings. No big deal, we can augment the method with a boolean parameter: void SimulateDay(int isSunday) { Console.WriteLine("Wake up."); Console.WriteLine(...

Stop doing this

Pet peeve: I hate micro-management when it comes to code. Yes, it's faster in the short-term but it's a pain when the code has to be changed. (If you don't know that all code has to be changed, you haven't programmed long enough.) In this particular case I have encountered code similar to: var sb = new StringBuilder(); foreach (var p in list) { sb.Append(p.Name); sb.Append(" "); sb.Append(p.Quantity); sb.Append("*"); sb.Append(p.Price); sb.Append("="); sb.Append(p.Value); sb.Append(","); } if (list.Any()) { sb.Remove(sb.Length - 1, 1); } return sb.ToString(); (It's even worse when it's a for loop instead of foreach , but let's not go there.) The problem with this code is that it's extremely easy to get lost in the details. "Oh, I should put spaces around the operations." "Oh, I should add a space after the comma, but then it has to be remov...

Append-only files

I need to implement an append-only persistent data structure for a project using CQRS (I will use it for the event store). The records will be variable in length (a string serialization of the events using either XML, JSON or YAML - I haven't decided yet). Now, appending to a text file is easy in Windows; just open the file for append. However, I will also need to read the events, starting with a given Id ; that's harder to do with a text file. Given that I'm in a "designing algorithms" mood lately (see my first Coursera certificate ) and that I haven't been able to find a satisfactory library written by someone else, here's what I came up with. The data file Given that the records in the data file will have different lengths, I could use an end-of-record character as a delimiter; however, finding the next record would require a full record scan in that case. (Plus, I am not 100% sure that I want to use text serialization - maybe I'll switch to b...