tag:blogger.com,1999:blog-56548762024-03-13T14:50:03.506+02:00Marcel PopescuMarcel Popescu, senior .NET programmerMarcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.comBlogger135125tag:blogger.com,1999:blog-5654876.post-36522580246906745392020-04-01T12:45:00.000+03:002020-04-01T12:46:36.831+03:00A generic database class<p>This is a class I use frequently for running SQL commands / queries:</p>
<pre style="font-family:JetBrains Mono;font-size:19px;color:gainsboro;background:#1e1e1e;"><span style="color:#569cd6;">public</span> <span style="color:#569cd6;">class</span> <span style="color:#727272;">Database</span> : <span style="color:lightblue;">IDatabase</span>
{
<span style="color:#569cd6;">public</span> <span style="color:lightblue;">Database</span>(<span style="color:lightblue;">DatabaseConfiguration</span> <span style="color:lime;">config</span>)
{
<span style="color:#569cd6;">this</span><span style="color:#b4b4b4;">.</span><span style="color:violet;">config</span> <span style="color:#b4b4b4;">=</span> <span style="color:lime;">config</span>;
}
<span style="color:#569cd6;">public</span> <span style="color:#569cd6;">object</span>[][] <span style="color:cyan;">RunQuery</span>(<span style="color:#569cd6;">string</span> <span style="color:lime;">query</span>, <span style="color:#569cd6;">params</span> <span style="color:#569cd6;">object</span>[] <span style="color:lime;">args</span>)
{
<span style="color:#569cd6;">using</span> (<span style="color:#569cd6;">var</span> <span style="color:lime;">con</span> <span style="color:#b4b4b4;">=</span> <span style="color:cyan;">Connect</span>())
<span style="color:#569cd6;">using</span> (<span style="color:#569cd6;">var</span> <span style="color:lime;">cmd</span> <span style="color:#b4b4b4;">=</span> <span style="color:cyan;">Prepare</span>(<span style="color:lime;">con</span>, <span style="color:lime;">query</span>, <span style="color:lime;">args</span>))
<span style="color:#569cd6;">return</span> <span style="color:lime;">cmd</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">ExecuteReader</span>()<span style="color:#b4b4b4;">.</span><span style="color:cyan;">Pipe</span>(<span style="color:cyan;">ReadData</span>);
}
<span style="color:#569cd6;">public</span> <span style="color:#569cd6;">void</span> <span style="color:cyan;">RunNonQuery</span>(<span style="color:#569cd6;">string</span> <span style="color:lime;">command</span>, <span style="color:#569cd6;">params</span> <span style="color:#569cd6;">object</span>[] <span style="color:lime;">args</span>)
{
<span style="color:#569cd6;">using</span> (<span style="color:#569cd6;">var</span> <span style="color:lime;">con</span> <span style="color:#b4b4b4;">=</span> <span style="color:cyan;">Connect</span>())
<span style="color:#569cd6;">using</span> (<span style="color:#569cd6;">var</span> <span style="color:lime;">cmd</span> <span style="color:#b4b4b4;">=</span> <span style="color:cyan;">Prepare</span>(<span style="color:lime;">con</span>, <span style="color:lime;">command</span>, <span style="color:lime;">args</span>))
<span style="color:lime;">cmd</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">ExecuteNonQuery</span>();
}
<span style="color:#57a64a;">//</span>
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">static</span> <span style="color:#569cd6;">readonly</span> <span style="color:lightblue;">ITableReader</span> <span style="color:violet;">TABLE_READER</span> <span style="color:#b4b4b4;">=</span> <span style="color:#569cd6;">new</span> <span style="color:lightblue;">TableReader</span>();
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">readonly</span> <span style="color:lightblue;">DatabaseConfiguration</span> <span style="color:violet;">config</span>;
<span style="color:#569cd6;">private</span> <span style="color:lightblue;">SqlConnection</span> <span style="color:cyan;">Connect</span>() <span style="color:#b4b4b4;">=></span>
<span style="color:#569cd6;">new</span> <span style="color:lightblue;">SqlConnection</span>(<span style="color:violet;">config</span><span style="color:#b4b4b4;">.</span><span style="color:violet;">ConnectionString</span>)<span style="color:#b4b4b4;">.</span><span style="color:cyan;">Apply</span>(<span style="color:lime;">it</span> <span style="color:#b4b4b4;">=></span> <span style="color:lime;">it</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">Open</span>());
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">static</span> <span style="color:#569cd6;">object</span>[][] <span style="color:cyan;">ReadData</span>(<span style="color:lightblue;">IDataReader</span> <span style="color:lime;">reader</span>) <span style="color:#b4b4b4;">=></span>
<span style="color:violet;">TABLE_READER</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">ReadData</span>(<span style="color:lime;">reader</span>);
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">static</span> <span style="color:lightblue;">SqlCommand</span> <span style="color:cyan;">Prepare</span>(<span style="color:lightblue;">SqlConnection</span> <span style="color:lime;">connection</span>, <span style="color:#569cd6;">string</span> <span style="color:lime;">sql</span>, <span style="color:#569cd6;">params</span> <span style="color:#569cd6;">object</span>[] <span style="color:lime;">args</span>) <span style="color:#b4b4b4;">=></span>
<span style="color:#569cd6;">new</span> <span style="color:lightblue;">SqlCommand</span>(<span style="color:lime;">sql</span>, <span style="color:lime;">connection</span>)<span style="color:#b4b4b4;">.</span><span style="color:cyan;">Apply</span>(<span style="color:lime;">it</span> <span style="color:#b4b4b4;">=></span> <span style="color:cyan;">AddParams</span>(<span style="color:lime;">it</span>, <span style="color:lime;">args</span>));
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">static</span> <span style="color:#569cd6;">void</span> <span style="color:cyan;">AddParams</span>(<span style="color:lightblue;">SqlCommand</span> <span style="color:lime;">cmd</span>, <span style="color:lightblue;">IEnumerable</span><<span style="color:#569cd6;">object</span>> <span style="color:lime;">args</span>) <span style="color:#b4b4b4;">=></span>
<span style="color:cyan;">ToParams</span>(<span style="color:lime;">args</span>)<span style="color:#b4b4b4;">.</span><span style="color:cyan;">Apply</span>(<span style="color:lime;">cmd</span><span style="color:#b4b4b4;">.</span><span style="color:violet;">Parameters</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">AddRange</span>);
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">static</span> <span style="color:lightblue;">SqlParameter</span>[] <span style="color:cyan;">ToParams</span>(<span style="color:lightblue;">IEnumerable</span><<span style="color:#569cd6;">object</span>> <span style="color:lime;">args</span>) <span style="color:#b4b4b4;">=></span>
<span style="color:lime;">args</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">Select</span>(<span style="color:cyan;">CreateParameter</span>)<span style="color:#b4b4b4;">.</span><span style="color:cyan;">ToArray</span>();
<span style="color:#569cd6;">private</span> <span style="color:#569cd6;">static</span> <span style="color:lightblue;">SqlParameter</span> <span style="color:cyan;">CreateParameter</span>(<span style="color:#569cd6;">object</span> <span style="color:lime;">value</span>, <span style="color:#569cd6;">int</span> <span style="color:lime;">index</span>) <span style="color:#b4b4b4;">=></span>
<span style="color:#569cd6;">new</span> <span style="color:lightblue;">SqlParameter</span>(<span style="color:cyan;">$</span><span style="color:#d69d85;">"@p</span>{<span style="color:lime;">index</span> <span style="color:#b4b4b4;">+</span> <span style="color:#b5cea8;">1</span>}<span style="color:#d69d85;">"</span>, <span style="color:lime;">value</span>);
}
</pre>
This is using two helper classes: the <code>TableReader</code> class
<pre style="font-family:JetBrains Mono;font-size:19px;color:gainsboro;background:#1e1e1e;"><span style="color:#569cd6;">internal</span> <span style="color:#569cd6;">class</span> <span style="color:lightblue;">TableReader</span> : <span style="color:lightblue;">ITableReader</span>
{
<span style="color:#569cd6;">public</span> <span style="color:#569cd6;">object</span>[][] <span style="color:cyan;">ReadData</span>(<span style="color:lightblue;">IDataReader</span> <span style="color:lime;">reader</span>)
{
<span style="color:#569cd6;">var</span> <span style="color:lime;">list</span> <span style="color:#b4b4b4;">=</span> <span style="color:#569cd6;">new</span> <span style="color:lightblue;">List</span><<span style="color:#569cd6;">object</span>[]>();
<span style="color:#569cd6;">while</span> (<span style="color:lime;">reader</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">Read</span>())
{
<span style="color:#569cd6;">var</span> <span style="color:lime;">values</span> <span style="color:#b4b4b4;">=</span> <span style="color:#569cd6;">new</span> <span style="color:#569cd6;">object</span>[<span style="color:lime;">reader</span><span style="color:#b4b4b4;">.</span><span style="color:violet;">FieldCount</span>];
<span style="color:lime;">reader</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">GetValues</span>(<span style="color:lime;">values</span>);
<span style="color:lime;">list</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">Add</span>(<span style="color:lime;">values</span>);
}
<span style="color:#569cd6;">return</span> <span style="color:lime;">list</span><span style="color:#b4b4b4;">.</span><span style="color:cyan;">ToArray</span>();
}
}
</pre>
and the <code>DatabaseConfiguration</code> class
<pre style="font-family:JetBrains Mono;font-size:19px;color:gainsboro;background:#1e1e1e;"><span style="color:#569cd6;">public</span> <span style="color:#569cd6;">class</span> <span style="color:lightblue;">DatabaseConfiguration</span>
{
<span style="color:#569cd6;">public</span> <span style="color:#569cd6;">string</span> <span style="color:violet;">ConnectionString</span> { <span style="color:cyan;">get</span>; <span style="color:cyan;">set</span>; }
}
</pre>
and also a couple of extension methods:
<pre style="font-family:JetBrains Mono;font-size:19px;color:gainsboro;background:#1e1e1e;"><span style="color:#569cd6;">public</span> <span style="color:#569cd6;">static</span> <span style="color:lightblue;">B</span> <span style="color:cyan;">Pipe</span><<span style="color:lightblue;">A</span>, <span style="color:lightblue;">B</span>>(<span style="color:#569cd6;">this</span> <span style="color:lightblue;">A</span> <span style="color:lime;">a</span>, <span style="color:lightblue;">Func</span><<span style="color:lightblue;">A</span>, <span style="color:lightblue;">B</span>> <span style="color:lime;">func</span>) <span style="color:#b4b4b4;">=></span> <span style="color:lime;">func</span>(<span style="color:lime;">a</span>);
<span style="color:#569cd6;">public</span> <span style="color:#569cd6;">static</span> <span style="color:lightblue;">A</span> <span style="color:cyan;">Apply</span><<span style="color:lightblue;">A</span>>(<span style="color:#569cd6;">this</span> <span style="color:lightblue;">A</span> <span style="color:lime;">a</span>, <span style="color:lightblue;">Action</span><<span style="color:lightblue;">A</span>> <span style="color:lime;">action</span>)
{
<span style="color:lime;">action</span>(<span style="color:lime;">a</span>);
<span style="color:#569cd6;">return</span> <span style="color:lime;">a</span>;
}</pre>
I am quite happy with the way this code looks - small classes and methods, everything should be pretty self-explanatory.
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-77476842512666018762020-03-14T15:10:00.002+02:002020-03-14T15:10:56.627+02:00The early bird catches the worm<p>Here are three reasons to do things later rather than earlier:</p>
<ul>
<li class="u">You will know more.</li>
<li class="u">You will usually be richer and thus can buy some help (at least for part of the task).</li>
<li class="u">You might be dead and therefore no longer care about the task :)
</li>
</ul>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-84909844676815805352019-07-09T13:29:00.000+03:002019-07-09T13:29:44.218+03:00On Dependency Injection<p>I just read an article about <a href="https://qualityisspeed.blogspot.com/2015/02/the-dependency-elimination-principle-a-canonical-example.html"> eliminating dependencies</a> whose basic thesis is that instead of having a class that depends on something that can give you a value, just depend on that value directly.</p>
<p>His example (which you can read more fully in the article) is that instead of having</p>
<pre>
public InvoiceGenerator(IConfigurationReader configurationReader)
{
_configurationReader = configurationReader;
}
</pre>
<p>and later on calling</p>
<pre>
var watermarkText = _configurationReader.Get<string>("invoiceWatermarkText");
if (!String.IsNullOrEmpty(watermarkText))
</pre>
<p>to get the text we need, just request the text directly:</p>
<pre>
public Invoice GenerateInvoice(string watermarkText)
{
if (!String.IsNullOrEmpty(watermarkText))
</pre>
<p>There are two obvious problems with this, which you will hit extremely quickly in real code.</p>
<p>One of them, pointed out by a comment by LeszekP, is that most of the time you will require more than a single value from an interface. Worse, some of them will only be required in specific cases (behind <code>if</code> branches).</p>
<p>The second one is that this method of eliminating the dependencies has actually increased them: instead of having one dependencies from the current class to the <code>IConfigurationReader</code> interface, we now have ALL clients of the current class suddenly requiring that dependency, because all of them now need to pass on the watermark value.</p>
<p>In short, although I would very much appreciate a clean way of reducing dependencies between classes, I don't believe this is the right method.</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-77208194550528605822019-07-08T06:21:00.001+03:002019-07-08T06:21:47.955+03:00My views on blockchain / bitcoin<p>Going from abstract to concrete:
</p><ul>
<li class="u">Private currency: good. I would love to have lots of them, just as I love both gold and silver.
</li>
<li class="u">Blockchain: not so good, more of a solution in search of a problem. The only advantage it has compared to an append-only database is decentralization… and I don't see a need for that. I would much rather have many private currencies. Also, in practice most coins are actually centralized, or at least a very small oligarchy.
</li>
<li class="u">Bitcoin: nah. Really bad. Incredibly slow to initialize, slow transactions, easy to control by a sustained state-level effort, very low cap on number of transactions per second, plenty of bugs, hard-forks which means the code is actually controlled by a small group (which in turn means that I can't trust that the currency itself is decentralized)… nah.</li></ul>
<p>In conclusion, I like the initiative, and I'm 100% behind the idea that we need to separate states and money, but we're nowhere near yet.</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com2tag:blogger.com,1999:blog-5654876.post-49736727112379434542019-04-14T03:17:00.000+03:002019-04-14T03:17:17.704+03:00The problem with null<p>The problem with <code>null</code> is that it pretends to be an object of a given type, without actually having that type. For example (C# code - ignore the uselessness of the <code>GetName</code> method):</p>
<pre>
string GetName(Customer customer)
{
// I got a customer object, I can access the Name property
return customer.Name;
}
var x = func(null); // <-- not a real Customer object so we have a run-time error
</pre>
<p>The proper way to solve this is by using the Option (aka Maybe) monad; for an example using the Functional.Maybe NuGet package:</p>
<pre>
string GetName(Maybe<Customer> customer)
{
// I don't actually have a Customer object, I have a Maybe<Customer>
// I need to treat it carefully
return customer.Select(it => it.Name).OrElse("Missing");
}
var x = func(null); // compiler error, because Maybe<> doesn't allow null as a value
var x = func(Maybe<Customer>.Nothing); // the customer is missing but the call will not crash
</pre>
<p>Using the Option monad is basically a way of forcing a compile-time check for <code>null</code>s, instead of waiting until run-time.</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-81900073032094372682018-10-01T22:17:00.001+03:002018-10-01T22:17:16.905+03:00FanFiction<h2>FanFiction</h2>
<p>Phrase from a book I'm reading: </p><blockquote><p>I was bitten by a radioactive Jedi as a child.</p></blockquote>
<p><a href="https://www.fanfiction.net/s/8501689/1/"> The Havoc side of the Force</a></p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-74112783190715061682017-12-30T16:03:00.000+02:002017-12-30T16:03:10.674+02:00Fan-fiction recommendations<p>I posted this to HN and I realized it might be useful for other people. If at least one person who wasn't aware of fan-fiction discovers it as a result, this has served its purpose.</p>
<p>For Buffy fans, <a href="https://www.tthfanfic.org"> TTH</a> has a huge number of stories, both in-universe and crossovers with other worlds. <a href="https://www.tthfanfic.org/Author-3980/Speakertocustomers.htm"> Speakertocustomers</a>, <a href="https://www.tthfanfic.org/Author-7859/becuzitswrong.htm"> Becuzitswrong</a>, <a href="https://www.tthfanfic.org/Author-15310/Cordyfan.htm"> Cordyfan</a>, <a href="https://www.tthfanfic.org/Author-22082/DianeCastle.htm"> DianeCastle</a> and <a href="https://www.tthfanfic.org/Author-7284/Hotpoint.htm"> Hotpoint</a> are a few of my favorite authors. Hotpoint's crossover between X-COM and SG-1 is absolutely amazing.</p>
<p>For other worlds, <a href="https://www.fanfiction.net"> FF</a>, <a href="https://forums.spacebattles.com/"> SB</a>, <a href="https://forums.sufficientvelocity.com"> SV</a> and <a href="http://archiveofourown.org/"> AO3</a> are the most well-known sites for fanfiction. One of my favorite stories is <a href="https://forums.sufficientvelocity.com/threads/taylor-varga-worm-luna-varga.32119/"> Taylor Varga</a>.</p>
<p>What I normally do is I download the stories in the .mobi format - TTH and AO3 have that feature in the site, and for the others you can use the FanFicFare plugin for Calibre - and then upload them to my Kindle. This allows me to read in bed, which is bad for my sleep but I like it :)
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-41565532677170333352017-03-31T11:43:00.001+03:002017-03-31T11:44:03.092+03:00Don't compare floats<p>Floating-point numbers are tricky; one of the first things a programmer needs to remember when working with them is to never check floats for equality. (I believe ReSharper warns you if you do that; I don't know if plain Visual Studio does because I never use it without ReSharper.)</p>
<p>If precise representations of decimal numbers is needed, like when manipulating currencies, use decimals instead (if using C#); if a similar primitive type does not exist in your language, write a separate package / module / library to "fake it" by using integer values and scale them down by two or four digits, depending on your needs. (For example, the number <code>123456</code> can represent either <code>1,234.56</code> or <code>12.3456</code>, depending on your application.)</p>
<p>Here's a simple code example to show the difference between <code>float</code>s and <code>decimal</code>s in C#:</p>
<pre>
float f1 = 0.1f;
float f2 = f1 * 10.0f;
float f3 = 0.0f;
for (var i = 1; i <= 10; i++)
f3 += f1;
Console.WriteLine(f2);
Console.WriteLine(f3);
Console.WriteLine(f2 == f3);
decimal d1 = 0.1m;
decimal d2 = d1 * 10.0m;
decimal d3 = 0.0m;
for (var i = 1; i <= 10; i++)
d3 += d1;
Console.WriteLine(d2);
Console.WriteLine(d3);
Console.WriteLine(d2 == d3);
</pre>
<p>Running this in LinqPad prints out:</p>
<pre>
1
1
False
1.00
1.0
True
</pre>
<p>(A peculiarity of the decimal type in C# is that it keeps track of the number of decimal places that have been used, hence the <code>1.00</code> vs <code>1.0</code> in the second part of the results.)
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-43379428327230493452017-03-15T16:42:00.001+02:002017-03-15T16:42:54.937+02:00Evaluating an expressionLeaving a note to myself - a simple algorithm for evaluating expressions, no unary operators or parentheses.
<pre>
phase 1 (tokenizer)
'+' => op1, add
'-' => op1, sub
'*' => op2, mul
'/' => op2, div
\d+ => number, value
phase 2 (evaluator)
tuple = first (number, op2, number)
while tuple:
replace tuple with result of op2
tuple = first (number, op1, number)
while tuple:
replace tuple with result of op1
there should be a single item left, a number; return its value
</pre>
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-80163936066996513082016-06-25T08:26:00.001+03:002016-06-25T08:26:26.711+03:00Returning an IEnumerable from a database<p>This is probably rather obvious, but... when returning an `IEnumerable` from a database, like the records from a table, don't do this:</p>
<pre>
using (var db = GetDatabase())
{
return db.GetTable("Table1");
}
</pre>
<p>because it will throw an exception when trying to enumerate those records, since the database connection has already been disposed.</p>
<p>Don't do this either:</p>
<pre>
using (var db = GetDatabase())
{
return db.GetTable("Table1").ToList();
}
</pre>
<p>because it will retrieve <strong>all</strong> records from the table, even if you only need a small subset.</p>
<p>A slightly better way is to do this:</p>
<pre>
using (var db = GetDatabase())
{
foreach (var item in db.GetTable("Table1"))
yield return item;
}
</pre>
<p>This way, the `Dispose` method doesn't get called until the enumeration is over and if you only `Take()` a limited number of records from the result, it won't load the whole table. On the other hand, if you only add `Where()` clauses to the result, it will still enumerate everything (the "where" doesn't get passed on to the database engine).</p>
<p>This is not perfect - it would be best if I could return an IQueryable so that additional filters get passed on to the database engine, but I haven't yet figured out how to do that.
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-55378461452246010762016-06-05T10:50:00.000+03:002016-06-05T10:50:03.219+03:00Aligning text<p>I needed to write some code in a console app to align what a user was saying, in case it was longer than a line (80 characters):</p>
<pre>
marcel: blah blah blah a lot of text that doesn't fit in 80 characters more blah
blah blah
long_username: Contrary to popular belief, Lorem Ipsum is not simply random text
. It has roots in a piece of classical Latin literature from 45 B
C, making it over 2000 years old. Richard McClintock, a Latin pro
fessor at Hampden-Sydney College in Virginia, looked up one of th
</pre>
<p>I wrote the code in two ways: an imperative, mutating style and a functional (recursive) style. I find the second one to be more elegant but since the entire project is something done as a hobby I don't much care about speed; your mileage might vary. I'm also quite certain the first method can be improved but… again, I don't need to do that so it doesn't get done.</p>
<p>As usual, use at your own risk, I don't care about copyrights, blah, blah. Oh, and the result will look bad if the text contains non-printable characters.</p>
<pre>
private static string Align(string prefix, string text, int length = 80)
{
Debug.Assert(prefix != null);
Debug.Assert(text != null);
Debug.Assert(length > prefix.Length);
var prefixLength = prefix.Length;
var sb = new StringBuilder();
do
{
var limit = Math.Min(text.Length, length - prefixLength);
var current = text.Substring(0, limit);
sb.Append(prefix + current);
prefix = new string(' ', prefixLength);
text = text.Substring(limit);
} while (text != "");
return sb.ToString();
}
private static string Align2(string prefix, string text, int length = 80)
{
Debug.Assert(prefix != null);
Debug.Assert(text != null);
Debug.Assert(length > prefix.Length);
var all = prefix + text;
return all.Length <= length
? all
: all.Substring(0, length) +
Align2(new string(' ', prefix.Length), all.Substring(length), length);
}
</pre>
<p>Hmm... I think this would be an interesting exercise if I wanted to hire programmers.</p>
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-57038462897083824422016-06-05T09:07:00.000+03:002016-06-05T09:07:29.318+03:00Math is a game<p>Math is a game, an arbitrary set of symbols and rules. The weird part, the part that always surprises me, is that it's relevant to the real world.</p>
<p>Here's an example: we'll start with just two symbols, Yin and Yang. (Replace those with black and white, circle and square, X and Y… whichever two symbols you prefer.) What can we do with them?</p>
<p>Well, the simplest thing we can do is transform one into the other:</p>
<p>Yin → Yang; Yang → Yin</p>
<p>We'll call this transformation "mirroring" and denote it with the letter "M".</p>
<p>What about combining two symbols? We have a number of possibilities:</p>
<p>A) Yin, Yang → Yin; Yang, Yin → Yin; Yin, Yin → Yin; Yang, Yang → Yin</p>
<p>This is rather boring… no matter what we start with, we obtain an Yin symbol. Nevertheless, let's continue.</p>
<p>B) The opposite of A: Yin, Yang → Yang; Yang, Yin → Yang; Yin, Yin → Yang; Yang, Yang → Yang. Still boring.</p>
<p>C) Yin, Yang → Yin; Yang, Yin → Yin; Yin, Yin → Yin; Yang, Yang → Yang</p>
<p>Ok, so this is a bit more interesting: this combination still overwhelmingly favors the Yin symbol, but at least once in a while we get back an Yang. For completion, we'll also write down its opposite:</p>
<p>D) Yin, Yang → Yang; Yang, Yin → Yang; Yin, Yin → Yang; Yang, Yang → Yin</p>
<p>The last two combinations give us an equal distribution of the results:</p>
<p>E) Yin, Yang → Yin; Yang, Yin → Yin; Yin, Yin → Yang; Yang, Yang → Yang</p>
<p>F) Yin, Yang → Yang; Yang, Yin → Yang; Yin, Yin → Yin; Yang, Yang → Yin</p>
<p>I'll summarize the A..F combinations in a table:</p>
<table border="1"><tr><td style="text-align:center;"> <strong>1st</strong> </td><td style="text-align:center;"> <strong>2nd</strong> </td><td style="text-align:center;"> <strong>A</strong> </td><td style="text-align:center;"> <strong>B</strong> </td><td style="text-align:center;"> <strong>C</strong> </td><td style="text-align:center;"> <strong>D</strong> </td><td style="text-align:center;"> <strong>E</strong> </td><td style="text-align:center;"> <strong>F</strong> </td></tr>
<tr><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td></tr>
<tr><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td></tr>
<tr><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td></tr>
<tr><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td><td style="text-align:center;"> Yang </td><td style="text-align:center;"> Yin </td></tr></table>
<p>So, what was the point of all this? In itself, nothing much. We can imagine that two kids might alternate playing this game, where one of them writes symbols and the other one combines them to obtain the result, and the one with the most correct results wins. It doesn't really matter: the interesting part is that we just re-invented Boolean logic. The Yin and Yang symbols are 0 and 1 (or <strong>true</strong> and <strong>false</strong>) and the transformations are the well-known Boolean operators: <strong>and</strong>, <strong>or</strong>, <strong>xor</strong>, <strong>equ</strong> and finally <strong>not</strong> for what we called "mirroring" (I've ignored the "boring" A and B). Given that Boolean logic is used by all computers, this is quite amazing - what looked like rather pointless playing with two symbols picked at random proved to be hugely important.
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-29977588833039068832016-06-02T17:32:00.000+03:002016-06-02T17:32:18.251+03:00Parsing INI files<p>I am trying to allow the end user to modify the application behavior in some limited ways; as such, I have a need of parsing .INI files of the form</p>
<pre>
[Type 1]
contains=abc
contains=def
[Type 2]
contains=1234
</pre>
<p>I looked around for a class / library that would allow me to read these files but I haven't found anything useful (most of the classes I found couldn't handle duplicate keys within the same section). As such, I went ahead and implemented this myself. This algorithm is extremely specific to my usage, you might have to modify it for your needs.</p>
<p>The <code>IniTuple</code> represents each line as a (section, key, value) tuple:</p>
<pre>
public class IniTuple
{
public string Section { get; private set; }
public string Key { get; private set; }
public string Value { get; private set; }
public IniTuple(string section, string key, string value)
{
Section = section;
Key = key;
Value = value;
}
}
</pre>
<p>The <code>IniParser</code> class implements the algorithm that transforms a string to a list of <code>IniTuple</code>s:</p>
<pre>
public class IniParser
{
public IEnumerable<IniTuple> Read(string text)
{
var lines = text
.Split('\r', '\n')
.Select(RemoveComment)
.Where(line => !string.IsNullOrWhiteSpace(line))
.ToArray();
var section = "";
foreach (var line in lines)
{
if (IsNewSection(line))
section = line.Substring(1, line.Length - 2);
else
yield return GetTuple(section, line);
}
}
//
private static string RemoveComment(string line)
{
var index = line.IndexOf(';');
return index < 0 ? line : line.Substring(0, index);
}
private static bool IsNewSection(string line)
{
return Regex.IsMatch(line, @"^\s*\[[^]]+\]\s*$");
}
private static IniTuple GetTuple(string section, string line)
{
var index = line.IndexOf('=');
return index < 0
? new IniTuple(section, "", line)
: new IniTuple(section, line.Substring(0, index), line.Substring(index + 1));
}
}
</pre>
<p>Please let me know if this was useful. As usual, I don't believe in copyrights so feel free to do whatever you want with the code.
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-11033507348583254152016-05-13T15:11:00.001+03:002016-05-13T15:11:31.790+03:00C# math is fast<p>I was reading an article on neural networks that mentioned the usual sigmoid activation function when the inputs are real numbers in the [0, 1) interval:</p>
<div class="eq">1 / (1 + e<sup class="i">−x</sup>)</div>
<p>The article mentions that this is probably where the program would spend at least half of its time so I thought "why not pre-compute a bunch of values and trade memory and precision for time"? It turns out, C# math is quite fast and the gain might not be worth it. (I haven't tested it yet with a NN.)</p>
<p>This is the code I wrote to benchmark the two options, using <a href="http://www.linqpad.net/"> LinqPad 5</a>:</p>
<pre>
void Main()
{
const int STEPS = 1 * 1000 * 1000;
Func<double, double> activation = x => 1.0 / (1.0 + Math.Exp(-x));
var cache = Precompute(0.0, 1.0, STEPS, activation);
Benchmark("Using the cache", x => cache[(int) Math.Truncate(x * STEPS)]);
Benchmark("Calling the function each time", activation);
}
double[] Precompute(double lower, double upper, int steps, Func<double, double> func)
{
var result = new List<double>();
For(0.0, 1.0, 1.0 / steps, value => result.Add(func(value)));
return result.ToArray();
}
void Benchmark(string message, Func<double, double> func)
{
const int COUNT = 100 * 1000 * 1000;
var dummy = 0.0;
var ts = ComputeBenchmark(() => For(0.0, 1.0, 1.0 / COUNT, value => dummy += func(value)));
Console.WriteLine(message + ": " + ts + " - sum = " + dummy);
}
void For(double lower, double upper, double increment, Action<double> action)
{
for (var value = lower; value <= upper; value += increment)
action(value);
}
TimeSpan ComputeBenchmark(Action action)
{
var sw = new Stopwatch();
sw.Start();
action();
sw.Stop();
return sw.Elapsed;
}
</pre>
<p>These are the results; using the cache takes about 70% of the time required when calling the function every time, at the expense of several MB of memory and some loss of precision. In a large NN, where billions of evaluations are expected in training, the trade-off might be worth it. I don't yet have enough experience to decide.</p>
<pre>
Using the cache: 00:00:01.9850792 - sum = 62011439.025702
Calling the function each time: 00:00:02.7755272 - sum = 62011450.5899971
</pre>
<p>As a side-note, I have computed and printed the sum to avoid the compiler from optimizing away the function calls; I don't know if LinqPad would do that but I wanted to avoid it just in case.
</p>
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-5525991391731273812016-04-19T19:01:00.002+03:002016-04-19T19:01:03.579+03:00A simple rules engine<p>I'm extracting data from some OCR'd letters and, in order to determine which type of letter I'm parsing, I'm using a method similar to this:</p>
<pre>
public Letter Parse(string text)
{
Letter result;
if (text.IndexOf("...", StringComparison.OrdinalIgnoreCase) >= 0)
letter = new LetterA();
else
letter = new LetterB();
//... additional processing
return letter;
}
</pre>
<p>If the letter contains a specific text, I know it's of one type; otherwise I'll default to the other type. Unfortunately that's going to get really complicated, really fast once I start adding new letter types. I read somewhere that "you should move logic out of the code and into the data when possible"; it made sense and I never had a reason to regret it. So, let me try to do that here.</p>
<p>First I'll add a "rules list" class that will allow me to store the various criteria:</p>
<pre>
public class RulesList<T, TResult>
{
public RulesList()
{
rules = new List<Tuple<Predicate<T>, Func<TResult>>>();
}
public void Add(Predicate<T> condition, Func<TResult> constructor)
{
rules.Add(Tuple.Create(condition, constructor));
}
public TResult Get(T criteria)
{
return rules
.Where(rule => rule.Item1(criteria))
.Select(rule => rule.Item2())
.FirstOrDefault();
}
//
private readonly List<Tuple<Predicate<T>, Func<TResult>>> rules;
}
</pre>
<p>I've made this class more generic by replacing the <code>string</code> type with <code>T</code>; honestly, I don't think there will ever be a need for anything else in this project but… it wasn't a big "expense".</p>
<p>Using this is quite simple at the moment:</p>
<pre>
public class LetterSelector
{
public LetterSelector()
{
rules = new RulesList<string, Letter>();
rules.Add(s => s.IndexOf("...", StringComparison.OrdinalIgnoreCase) >= 0, () => new LetterA());
rules.Add(_ => true, () => new LetterB());
}
public Letter Parse(string text)
{
var letter = rules.Get(text);
//... additional processing
return letter;
}
//
private readonly RulesList<string, Letter> rules;
}
</pre>
<p>Is this a big gain? Right now it doesn't look like I gained anything; however, bitter experience taught me that methods with many conditionals quickly become an unmaintainable mess (you haven't lived until you've had to fix a method with 700+ lines and a cyclomatic complexity over 200). This will allow me to separate those conditionals into their own lambdas or small private methods in the <code>LetterSelector</code> class.</p>
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-87041582218596801432016-04-12T14:13:00.005+03:002016-04-12T14:13:50.495+03:00Crystal Reports woes<p>This took me an hour to figure out so I thought I'd write it down in case it helps anyone else.</p>
<p>If you have a form that's going to display a Crystal Report and you want to zoom it by default, the "normal" way would be to do this in form_Shown:</p>
<pre>
private void ReportViewer_Shown(object sender, EventArgs e)
{
viewer.Zoom(2); // 1 = page width, 2 = whole page, 25..400 = zoom factor
}
</pre>
<p>(Where <code>viewer</code> is the <code>CrystalReportViewer</code> component.)</p>
<p>Unfortunately, it takes CR a while to compute and display the actual report; by the time that happens, the <code>.Zoom()</code> call has already been executed (and ignored).</p>
<p>I have tried a number of workarounds (including launching a thread, waiting for two seconds and then calling the <code>Zoom</code> method - it worked but it was a horrible hack) before I discovered that CR has a "hidden" <code>PageChanged</code> event (it has a <code>[Browsable(false)]</code> attribute). Use that event by assigning a handler in the constructor:</p>
<pre>
viewer.PageChanged += Viewer_PageChanged;
</pre>
<p>and then add the <code>Viewer_PageChanged</code> method:</p>
<pre>
private void Viewer_PageChanged(object sender, EventArgs e)
{
viewer.Zoom(2);
viewer.PageChanged -= Viewer_PageChanged;
}
</pre>
<p>(Note that, in order to avoid leaking references, I have removed the <code>PageChanged</code> handler immediately after calling the <code>Zoom</code> method.)
</p>
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com2tag:blogger.com,1999:blog-5654876.post-59668925000363362302016-03-26T21:35:00.000+02:002016-03-26T21:35:56.553+02:00POST-ing to an ASP.NET MVC form with an anti-forgery token<p>I've had some issues trying to write an acceptance test that was registering a new user by POST-ing the required information to a MVC site and I got back these messages:
</p><ul>
<li class="u">The required anti-forgery cookie "__RequestVerificationToken" is not present.</li>
<li class="u">The required anti-forgery form field "__RequestVerificationToken" is not present.</li>
<li class="u">Validation of the provided anti-forgery token failed. The cookie "_RequestVerificationToken" and the form field "_RequestVerificationToken" were swapped.</li></ul>
<p>Since it took me a bit of fiddling with the code before I managed to make it work, I thought I'd share the solution. The site is an ASP.NET MVC version 5 and I am trying to register a new user (POST-ing to the /Account/Register URL). The main issue you will encounter is having to extract <strong>two</strong> anti-forgery tokens, one from the cookies and one from the form, and then sending both of them in the appropriate places (cookies vs form field).</p>
<p>I have used LinqPad 5 with a reference to the <a href="https://www.nuget.org/packages/HtmlAgilityPack">HtmlAgilityPack</a> NuGet package (to simplify extracting the needed information from the GET form). This is the code:</p>
<pre>
void Main()
{
Cookie[] cookies;
var html = Get("Account/Register", out cookies);
var root = Parse(html);
var formToken = GetFormToken(root);
var cookieToken = GetCookieToken(cookies);
var unique = Guid.NewGuid().ToString("N");
Post("Account/Register", new
{
Email = unique + "@example.com",
User = unique,
Password = unique,
ConfirmPassword = unique,
},
formToken, cookieToken);
}
private const string BASE_URL = "http://localhost:5972"; // replace as needed
private const string TOKEN_NAME = "__RequestVerificationToken";
private static string Get(string url, out Cookie[] responseCookies)
{
using (var web = new CookieAwareWebClient())
{
var result = web.DownloadString(BASE_URL + "/" + url);
responseCookies = web.ResponseCookies.Cast<Cookie>().ToArray();
return result;
}
}
private static string Post(string url, object body, string formToken = null, string cookieToken = null)
{
using (var web = new WebClient())
{
var data = GetPostData(body);
web.Headers["Content-Type"] = "application/x-www-form-urlencoded";
if (formToken != null)
data += "&" + TOKEN_NAME + "=" + formToken;
if (cookieToken != null)
web.Headers.Add(HttpRequestHeader.Cookie, TOKEN_NAME + "=" + cookieToken);
return web.UploadString(BASE_URL + "/" + url, data);
}
}
private static string GetPostData(object obj)
{
var kv = obj
.GetType()
.GetProperties(BindingFlags.Instance | BindingFlags.Public)
.Select(prop => prop.Name + "=" + WebUtility.UrlEncode(prop.GetValue(obj) + ""))
.ToArray();
return string.Join("&", kv);
}
private static HtmlNode Parse(string html)
{
var doc = new HtmlDocument();
doc.LoadHtml(html);
return doc.DocumentNode;
}
private static string GetFormToken(HtmlNode root)
{
var formToken = root.SelectSingleNode("//*[@name='" + TOKEN_NAME + "']");
return formToken.GetAttributeValue("value", null);
}
private static string GetCookieToken(IEnumerable<Cookie> cookies)
{
return cookies
.Where(it => it.Name == TOKEN_NAME)
.First()
.Value;
}
// from http://stackoverflow.com/a/29479390
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; }
public CookieCollection ResponseCookies { get; set; }
public CookieAwareWebClient()
{
CookieContainer = new CookieContainer();
ResponseCookies = new CookieCollection();
}
protected override WebRequest GetWebRequest(Uri address)
{
var request = (HttpWebRequest) base.GetWebRequest(address);
// ReSharper disable once PossibleNullReferenceException
request.CookieContainer = CookieContainer;
return request;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
var response = (HttpWebResponse) base.GetWebResponse(request);
// ReSharper disable once PossibleNullReferenceException
ResponseCookies = response.Cookies;
return response;
}
}
</pre>
<p>Hope this helps someone.
</p>
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-12493522162028282252016-01-11T08:51:00.001+02:002016-01-11T08:51:15.228+02:00Processing a downloaded text file while it's downloading<p>I've had a job where I had to download some huge text files and I thought it interesting to process them <strong>while</strong> they are downloading, instead of waiting until the download was finished. (Time was very important.) Ultimately, the client changed his mind and wanted the whole thing downloaded first but I thought this was an interesting code fragment to save for later.</p>
<p>When it comes to "do real-time stuff" I use the <a href="https://www.nuget.org/packages/Rx-Main/"> Rx-Main NuGet package</a>. The interface is simple:</p>
<pre>
public interface Downloader
{
/// <summary>
/// Downloads a file from the given URL and returns it line by line.
/// </summary>
/// <param name="url">The URL of the file to be downloaded.</param>
/// <returns>The lines from the downloaded file, as a stream.</returns>
IObservable<string> Download(string url);
}
</pre>
<p>The only noteworthy thing about the implementation is the automatic decompression; using that property sends the appropriate headers to the server (do <strong>not</strong> send them separately, they will be duplicated and some servers don't handle that well):</p>
<pre>
public class WebDownloader : Downloader
{
public IObservable<string> Download(string url)
{
return Observable.Create<string>(o =>
{
var req = (HttpWebRequest) WebRequest.Create(url);
req.Method = WebRequestMethods.Http.Get;
req.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
using (var res = req.GetResponse())
using (var stream = res.GetResponseStream())
using (var reader = new StreamReader(stream))
{
string line;
while ((line = reader.ReadLine()) != null)
o.OnNext(line);
o.OnCompleted();
}
return Disposable.Empty;
});
}
}
</pre>
<p>That's pretty much all there is to it. Using this class is also simple:</p>
<pre>
downloader
.Download(url)
.ObserveOn(Scheduler.Default)
.Skip(1)
.Select(lineProcessor.Process)
.Where(it => it != null)
...
</pre>
<p>I am using <code>ObserveOn</code> to let processing be done on another thread, concurrently with the download, <code>Skip(1)</code> to skip the header line and the <code>Where</code> clause to skip any processing errors.
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-35092990183441015902015-11-14T09:28:00.000+02:002015-11-14T09:28:38.125+02:00A data flow helper class<p>One problem I encounter when processing lists is exception handling. I prefer to write code that "chains" calls transforming the data:</p>
<pre>
var results = list
.Select(DoThing1)
.Select(DoThing2)
// ...
.Select(DoThingN)
.ToList();
</pre>
<p>The problem with something like this is that, if any of the calls throws an exception, processing stops for the whole list. Handling that requires that I move the "chain" to a new method and handle exceptions there:</p>
<pre>
var results = list.Select(InnerMethod).ToList();
// ...
private ResultN InnerMethod(Input input)
{
try
{
var r1 = DoThing1(input);
var r2 = DoThing2(r1);
// ...
var rn = DoThingN(rn_1);
return rn;
}
catch(Exception ex)
{
// do something with ex, like logging
return ?? // can't throw, I want to continue processing the rest of the list
}
}
</pre>
<p>Now I have two problems :) One is that the code just looks uglier, so maybe most people can ignore that. (I have OCD with regards to this - code that "looks bad" drives me nuts.) The more important issue is that I need to decide on an "empty" ResultN value to return from the inner method and then I need to be able to filter those out of the overall results. That can get ugly really quickly.</p>
<p>By analogy with what I read about other languages (I think GO uses this approach - I've never studied the language but I believe I first encountered the idea in some articles about it), I decided to write an "either a good value or an exception" helper class. On further reflection I changed that to a <code>struct</code> because I don't want to check that the value is not <code>null</code>. Once I thought of <strong>that</strong> I also decided that <code>null</code> is not a "good value", so any attempt to pass it as such will result in an <code>ArgumentNullException</code> instead in the "or an exception" part. I hope things will become clearer from the code:</p>
<pre>
public struct Result<T>
{
public bool HasValue { get; }
public T Value
{
get
{
if (!HasValue)
throw Exception;
return value;
}
}
public Exception Exception => HasValue ? null : exception ?? NULL_EXCEPTION;
public Result(T value)
{
// do not accept null
if (value == null)
{
HasValue = false;
this.value = default(T);
exception = new ArgumentNullException();
}
else
{
HasValue = true;
this.value = value;
exception = null;
}
}
public Result(Exception ex)
{
HasValue = false;
value = default(T);
exception = ex;
}
//
// ReSharper disable once StaticMemberInGenericType
private static readonly Exception NULL_EXCEPTION = new ArgumentNullException();
private readonly T value;
private readonly Exception exception;
}
</pre>
<p>I also added an <code>Apply</code> extension method - the reason it's an extension method instead of a method in the original <code>struct</code> is because of the additional generic type <code>TR</code>; it just looked wrong there. (I did mention my OCD, right?)</p>
<pre>
public static class ResultExtensions
{
public static Result<TR> Apply<T, TR>(this Result<T> it, Func<T, TR> selector)
{
return it.HasValue
? Try(() => selector(it.Value))
: new Result<TR>(it.Exception);
}
public static IEnumerable<Result<TR>> Select<T, TR>(this IEnumerable<Result<T>> list, Func<T, TR> selector)
{
return list.Select(it => it.Apply(selector));
}
//
private static Result<TR> Try<TR>(Func<TR> func)
{
try
{
return new Result<TR>(func());
}
catch (Exception ex)
{
return new Result<TR>(ex);
}
}
}
</pre>
<p>While I didn't write this in a TDD fashion, I have added some asserts to a console application to make sure I got back the expected results:</p>
<pre>
static class Program
{
static void Main()
{
var r1 = Divide(5, 2);
Debug.Assert(Print(r1) == "HasValue = True Value = 2 Exception = ");
var r2 = Divide(5, 0);
Debug.Assert(Print(r2) == "HasValue = False Value = (invalid) Exception = Attempted to divide by zero.");
var r3 = new Result<int?>(3);
Debug.Assert(Print(r3) == "HasValue = True Value = 3 Exception = ");
var r4 = new Result<int?>((int?) null);
Debug.Assert(Print(r4) == "HasValue = False Value = (invalid) Exception = Value cannot be null.");
var r5 = new Result<object>(null);
Debug.Assert(Print(r4) == "HasValue = False Value = (invalid) Exception = Value cannot be null.");
// using the default constructor
var r6 = new Result<int>();
Debug.Assert(Print(r6) == "HasValue = False Value = (invalid) Exception = Value cannot be null.");
// trying to access the Value property without checking first will result in an exception
try
{
Console.WriteLine(r5.Value);
}
catch
{
Console.WriteLine("Oops.");
}
// we can now chain selectors
// case 1: all good
var i1 = new Result<int?>(5);
var ri1 = i1
.Apply(it => 100 / it)
.Apply(it => 200 / it)
.Apply(it => 10 / it);
Debug.Assert(Print(ri1) == "HasValue = True Value = 1 Exception = ");
// case 2: something bad happens
var i2 = new Result<int>(200);
var ri2 = i2
.Apply(it => 100 / it)
.Apply(it => 200 / it)
.Apply(it => 10 / it);
Debug.Assert(Print(ri2) == "HasValue = False Value = (invalid) Exception = Attempted to divide by zero.");
// finally, the target use case: processing a list without aborting due to exceptions
var list1 = new List<int> { 10, 20, 0, 30, 40 };
var list2 = list1
.Select(it => new Result<int>(it))
.Select(it => it / 2)
.Select(it => 10 / it)
.Select(it => 100 / it)
.ToList();
var good = list2.Where(it => it.HasValue).ToList();
var bad = list2.Where(it => !it.HasValue).ToList();
Debug.Assert(good.Count == 2);
Debug.Assert(bad.Count == 3);
}
private static Result<int> Divide(int a, int b)
{
try
{
return new Result<int>(a / b);
}
catch (Exception ex)
{
return new Result<int>(ex);
}
}
private static string Print<T>(Result<T> r)
{
return $"HasValue = {r.HasValue} Value = {(r.HasValue ? r.Value + "" : "(invalid)")} Exception = {r.Exception?.Message}";
}
}
</pre>
<p>Note that the addition of the <code>Select</code> extension method, I didn't have to write the last example as</p>
<pre>
var list2 = list1
.Select(it => new Result<int>(it))
.Select(it => it.Apply(x => x / 2))
.Select(it => it.Apply(x => 10 / x))
.Select(it => it.Apply(x => 100 / x))
.ToList();
</pre>
<p>Avoiding boilerplate code is good; so is the fact that the inner lambda doesn't have to know anything about the <code>Result<T></code> type and yet, any crash in it doesn't abort processing the entire list.
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-80503087112259826092015-10-20T21:46:00.001+03:002016-07-24T08:09:08.591+03:00Retry algorithm<h3> Retry with exponential back-off </h3>
<p><i>Update on July 24, 2016:</i> I just discovered <a href="https://github.com/App-vNext/Polly">Polly</a>, which does this and more a lot better.</p>
<p>I think this is an useful class so I'm just going to leave it here. (I'm annoyed by the duplication between <code>Retry</code> and <code>RetryAsync</code> but I haven't been able to remove it.)</p>
<pre>
public interface RetryPolicy
{
T Retry<T>(Func<T> func);
void Retry(Action action);
Task<T> RetryAsync<T>(Func<Task<T>> func);
Task RetryAsync(Func<Task> action);
}
public class RetryPolicyWithExponentialDelay : RetryPolicy
{
// ReSharper disable once InconsistentNaming
public Func<double> GetRandom = () => RND.NextDouble();
// ReSharper disable once InconsistentNaming
public Action<int> Sleep = timeout => Thread.Sleep(timeout);
public RetryPolicyWithExponentialDelay(int maxCount, TimeSpan initialDelay, TimeSpan maxDelay)
{
this.maxCount = maxCount;
this.initialDelay = initialDelay;
this.maxDelay = maxDelay;
}
public T Retry<T>(Func<T> func)
{
var count = 0;
var delay = initialDelay;
while (true)
{
try
{
return func();
}
catch
{
count++;
if (count >= maxCount)
throw;
SleepUpTo(delay);
delay = IncreaseDelay(delay);
}
}
}
public void Retry(Action action)
{
Retry(() =>
{
action();
return 0;
});
}
public async Task<T> RetryAsync<T>(Func<Task<T>> func)
{
var count = 0;
var delay = initialDelay;
while (true)
{
try
{
return await func();
}
catch
{
count++;
if (count >= maxCount)
throw;
SleepUpTo(delay);
delay = IncreaseDelay(delay);
}
}
}
public async Task RetryAsync(Func<Task> action)
{
await RetryAsync(async () =>
{
await action();
return 0;
});
}
//
private static readonly Random RND = new Random();
private readonly int maxCount;
private readonly TimeSpan initialDelay;
private readonly TimeSpan maxDelay;
private void SleepUpTo(TimeSpan delay)
{
var actualDelay = (int) Math.Truncate(GetRandom() * delay.TotalMilliseconds);
Sleep(actualDelay);
}
private TimeSpan IncreaseDelay(TimeSpan delay)
{
delay = delay.Add(delay);
if (delay > maxDelay)
delay = maxDelay;
return delay;
}
}
</pre>
<p>I'm also adding the tests here:</p>
<pre>
[TestClass]
public class RetryPolicyWithExponentialDelayTests
{
private const int RESULT = 100;
private RetryPolicyWithExponentialDelay sut;
private int called;
private int sleepTime;
[TestInitialize]
public void SetUp()
{
sut = new RetryPolicyWithExponentialDelay(5, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(5))
{
GetRandom = () => 1.0,
Sleep = timeout => sleepTime += timeout,
};
called = 0;
sleepTime = 0;
}
[TestClass]
public class Sync : RetryPolicyWithExponentialDelayTests
{
private Action action;
private Func<int> func;
[TestInitialize]
public void InnerSetup()
{
action = null;
func = () =>
{
action();
return RESULT;
};
}
[TestClass]
public class SyncFunc : Sync
{
[TestMethod]
public void NoErrors()
{
action = GetActionWithErrors(0);
var result = sut.Retry(func);
Assert.AreEqual(1, called, "Function was not called");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(0, sleepTime, "Should not have slept");
}
[TestMethod]
public void OneError()
{
action = GetActionWithErrors(1);
var result = sut.Retry(func);
Assert.AreEqual(2, called, "The call was not retried");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(TimeSpan.FromSeconds(1).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public void TwoErrors()
{
action = GetActionWithErrors(2);
var result = sut.Retry(func);
Assert.AreEqual(3, called, "The call was not retried twice");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public void FourErrors()
{
action = GetActionWithErrors(4);
var result = sut.Retry(func);
Assert.AreEqual(5, called, "The call was not retried four times");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
[TestMethod]
public void TooManyErrors()
{
action = GetActionWithErrors(10);
try
{
sut.Retry(func);
Assert.Fail("The call did not throw");
}
catch
{
Assert.AreEqual(5, called, "The call was not tried five times");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
}
}
[TestClass]
public class SyncAction : Sync
{
[TestMethod]
public void NoErrors()
{
action = GetActionWithErrors(0);
sut.Retry(action);
Assert.AreEqual(1, called, "Function was not called");
Assert.AreEqual(0, sleepTime, "Should not have slept");
}
[TestMethod]
public void OneError()
{
action = GetActionWithErrors(1);
sut.Retry(action);
Assert.AreEqual(2, called, "The call was not retried");
Assert.AreEqual(TimeSpan.FromSeconds(1).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public void TwoErrors()
{
action = GetActionWithErrors(2);
sut.Retry(action);
Assert.AreEqual(3, called, "The call was not retried twice");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public void FourErrors()
{
action = GetActionWithErrors(4);
sut.Retry(action);
Assert.AreEqual(5, called, "The call was not retried four times");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
[TestMethod]
public void TooManyErrors()
{
action = GetActionWithErrors(10);
try
{
sut.Retry(action);
Assert.Fail("The call did not throw");
}
catch
{
Assert.AreEqual(5, called, "The call was not tried five times");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
}
}
}
[TestClass]
public class Async : RetryPolicyWithExponentialDelayTests
{
private Action action;
private Func<Task<int>> func;
[TestInitialize]
public void InnerSetup()
{
action = null;
func = async () =>
{
await Task.Run(action);
return RESULT;
};
}
[TestClass]
public class AsyncFunc : Async
{
[TestMethod]
public async Task NoErrorsAsync()
{
action = GetActionWithErrors(0);
var result = await sut.RetryAsync(func);
Assert.AreEqual(1, called, "Function was not called");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(0, sleepTime, "Should not have slept");
}
[TestMethod]
public async Task OneErrorAsync()
{
action = GetActionWithErrors(1);
var result = await sut.RetryAsync(func);
Assert.AreEqual(2, called, "The call was not retried");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(TimeSpan.FromSeconds(1).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public async Task TwoErrorsAsync()
{
action = GetActionWithErrors(2);
var result = await sut.RetryAsync(func);
Assert.AreEqual(3, called, "The call was not retried twice");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public async Task FourErrorsAsync()
{
action = GetActionWithErrors(4);
var result = await sut.RetryAsync(func);
Assert.AreEqual(5, called, "The call was not retried four times");
Assert.AreEqual(RESULT, result, "Invalid result");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
[TestMethod]
public async Task TooManyErrorsAsync()
{
action = GetActionWithErrors(10);
try
{
await sut.RetryAsync(func);
Assert.Fail("The call did not throw");
}
catch
{
Assert.AreEqual(5, called, "The call was not tried five times");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
}
}
[TestClass]
public class AsyncAction : Async
{
[TestMethod]
public async Task NoErrorsAsync()
{
action = GetActionWithErrors(0);
await sut.RetryAsync(() => Task.Run(action));
Assert.AreEqual(1, called, "Function was not called");
Assert.AreEqual(0, sleepTime, "Should not have slept");
}
[TestMethod]
public async Task OneErrorAsync()
{
action = GetActionWithErrors(1);
await sut.RetryAsync(() => Task.Run(action));
Assert.AreEqual(2, called, "The call was not retried");
Assert.AreEqual(TimeSpan.FromSeconds(1).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public async Task TwoErrorsAsync()
{
action = GetActionWithErrors(2);
await sut.RetryAsync(() => Task.Run(action));
Assert.AreEqual(3, called, "The call was not retried twice");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount");
}
[TestMethod]
public async Task FourErrorsAsync()
{
action = GetActionWithErrors(4);
await sut.RetryAsync(() => Task.Run(action));
Assert.AreEqual(5, called, "The call was not retried four times");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
[TestMethod]
public async Task TooManyErrorsAsync()
{
action = GetActionWithErrors(10);
try
{
await sut.RetryAsync(() => Task.Run(action));
Assert.Fail("The call did not throw");
}
catch
{
Assert.AreEqual(5, called, "The call was not tried five times");
Assert.AreEqual(TimeSpan.FromSeconds(1 + 2 + 4 + 5).TotalMilliseconds, sleepTime, 1, "Did not sleep the correct amount (limited by max delay)");
}
}
}
}
//
private Action GetActionWithErrors(int errorCount)
{
return () =>
{
called++;
if (called <= errorCount)
throw new Exception();
};
}
}
</pre>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-12357304491695846442014-05-30T16:24:00.000+03:002014-05-30T17:26:21.301+03:00Stupid code fragments, part two<p>Probabilities are hard. As an example, there's a known puzzle: a family has two children; if one of them is a girl, what is the probability that the other one is also a girl?</p>
<p>The answer, un-intuitively, is <strong>not</strong> 1/2 but 1/3. There are various explanations but – as with the Monty Python puzzle years ago – I wanted to write code to check it out, so I wrote the following using LinqPad:</p>
<pre>
void Main()
{
var rnd = new Random();
// Generate a random set of families with two children; true means girl, false means boy
var all = Enumerable.Range(1, 10000).Select(_ => new Pair(rnd.Next(2) == 0, rnd.Next(2) == 0)).ToList();
// Extract only the families with at least one girl
var oneGirl = all.Where(it => it.First || it.Second).ToList();
// Out of those families, how many have two girls? The result should be 1/3rd
var otherGirl = oneGirl.Where(it => it.First && it.Second).ToList();
Console.WriteLine((decimal) otherGirl.Count / oneGirl.Count);
}
// Define other methods and classes here
public class Pair
{
public bool First { get; private set; }
public bool Second { get; private set; }
public Pair(bool first, bool second)
{
First = first;
Second = second;
}
}
</pre>
<p>Conditional probabilities (that is, probabilities where we have additional information – in this case, knowing that one child is a girl) are surprisingly tricky. A way to express the condition more clearly is: what is the probability that a family with two kids has two girls, <strong>given that</strong> they have at least one girl? Probabilities are, at base, expressions of uncertainty. If we ask "what is the probability that a family with two children has two girls?" we're in a situation of maximum uncertainty: all outcomes (BB, BG, GB and GG) are equally probable so the best we can do is 1/4 – that is the mathematical equivalent of "no clue". However, if we add some information – namely that one of the children is definitely a girl – then we removed some uncertainty from the problem: the BB case is no longer possible. This lowers the uncertainty of the GG case to 1/3.</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-32840275107192799032014-05-30T16:12:00.003+03:002014-05-30T16:12:48.826+03:00Stupid code fragments, part one<p>I just discovered a surprisingly simple (and obvious in hindsight) algorithm for calculating the week index of a given date. For example, April 15th is in the 3rd week (or the 3rd Tuesday of the month).</p>
<p>I was going to do the usual thing and just Google for it but then I realized that the solution is extremely simple:</p>
<pre>
private static int GetWeekPosition(DateTime date)
{
// the position of the given date is how many times I can subtract 7 days (go back one week) and still be in the same month
// in other words, it's the integer part of (day / 7)
return date.Day / 7;
}
</pre>
<p>(I am returning a base-zero result, but you can of course add 1 if you need it.)</p>
<p>I realize this is not the answer to the Universe or anything but I thought it's interesting.</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-49117034872046041512014-05-15T17:50:00.000+03:002014-05-15T17:50:20.894+03:00ScienceI've long used the expression "real science, so called because it can only be found in books and movies". From a <a href="http://www.pyvideo.org/video/2683/keynote-fernando-perez">talk at Pycon 2014</a>:
<blockquote>
The <strike>ideals</strike> reality of science:
<ul>
<li>The pursuit of <strike>verifiable answers</strike> highly cited papers for your c.v.</li>
<li>The validation of our results by <strike>reproduction</strike> convincing referees who did not see your code or data</li>
<li><strike>An altruistic, collective enterprise</strike> A race to outrun your colleagues in front of the giant bear of grant funding</li>
</ul>
</blockquote>
H.T to <a href="http://lemire.me/blog/archives/2014/04/14/science-ideals-vs-reality/">Daniel Lemire</a>.
Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0tag:blogger.com,1999:blog-5654876.post-70753617458005685792013-10-31T00:15:00.000+02:002013-10-31T00:15:19.184+02:00Telerik components<p>I am working on some projects where I am forced to use the Telerik components. They are, without a doubt, the most overpriced junk I've ever had to work with. It pains me that people pay a lot of money for them just because they look pretty. (I am trying to remember ONE issue where the Telerik components were involved and I haven't had to fight them. It's just not coming to mind.)</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com2tag:blogger.com,1999:blog-5654876.post-54117647697853414952013-10-05T23:28:00.000+03:002013-10-05T23:28:27.769+03:00Avoid boolean parameters<p>Here's an example of a method in a sim. This started out as a simple method: all days are the same in our simulation, so the method just prints out some activities:</p>
<pre>
void SimulateDay()
{
Console.WriteLine("Wake up.");
Console.WriteLine("Eat breakfast.");
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat lunch.");
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat dinner.");
Console.WriteLine("Go to sleep.");
}
</pre>
<p>Ok, so not much of a life, but this article is supposed to be about programming. :)</p>
<p>A new requirement comes up: Sunday should be different. Well, almost; it's actually the same as the other days with a single difference: instead of watching TV, the sim goes to church on Sunday mornings. No big deal, we can augment the method with a boolean parameter:</p>
<pre>
void SimulateDay(int isSunday)
{
Console.WriteLine("Wake up.");
Console.WriteLine("Eat breakfast.");
if (isSunday)
Console.WriteLine("Go to church.");
else
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat lunch.");
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat dinner.");
Console.WriteLine("Go to sleep.");
}
</pre>
<p>This works, no doubt about it, but it's a bad idea because the reasons to change this method are different. The sim might get a job; that will affect the work days but not the Sunday. He might become an atheist and stop going to church on Sundays. The two cases are conceptually different.</p>
<p>That means that, for the sake of maintainability, it pays to use two different methods:</p>
<pre>
void SimulateWeekDay()
{
Console.WriteLine("Wake up.");
Console.WriteLine("Eat breakfast.");
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat lunch.");
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat dinner.");
Console.WriteLine("Go to sleep.");
}
void SimulateSunday()
{
Console.WriteLine("Wake up.");
Console.WriteLine("Eat breakfast.");
Console.WriteLine("Go to church.");
Console.WriteLine("Eat lunch.");
Console.WriteLine("Watch TV.");
Console.WriteLine("Eat dinner.");
Console.WriteLine("Go to sleep.");
}
</pre>
<p>and decide at the time of the call which one to use. (I will admit that this contradicts the "defer decisions" principle, but I believe that one is more about design and not about actual <code>if</code>s in the implementation.)</p>
<p>Of course, the methods as they are now have a lot of duplicate code; those can be extracted into common methods, as long as it makes sense; for example:</p>
<pre>
void SimulateWeekDay()
{
Morning();
Console.WriteLine("Watch TV.");
Afternoon();
Evening();
}
void SimulateSunday()
{
Morning();
Console.WriteLine("Go to church.");
Afternoon();
Evening();
}
void Morning()
{
Console.WriteLine("Wake up.");
Console.WriteLine("Eat breakfast.");
}
void Afternoon()
{
Console.WriteLine("Eat lunch.");
Console.WriteLine("Watch TV.");
}
void Evening()
{
Console.WriteLine("Eat dinner.");
Console.WriteLine("Go to sleep.");
}
</pre>
<p>(I realize that refactoring some dumb example is really OCD. Hey, I really hate code duplication, ok?)</p>
<p>For the same reason, I would prefer to avoid a method like <code>DoStuff(..., bool withLogging)</code> and just pass a <code>Logger</code> to it instead. Of course, that won't gain me much if instead of peppering the method with <code>if (withLogging)</code> statements I replace them with <code>if (logger != null)</code>; a <code>NullLogger</code> that just doesn't do anything would be much better in this case. (Care must be taken so that the expressions sent to the logger are side-effect free, of course, but that's a good idea in itself.)</p>
<p>I've been doing this for a while and the only counter-example I've been able to find is when the booleans are passed on to something else. For example, earlier today I wrote code like this:</p>
<pre>
var ck = (bool) item.Value;
SetEnabled("items", ck);
SetEnabled("sqlquery", !ck);
</pre>
<p>where the <code>SetReadOnlyState</code> method sets the <code>Enabled</code> property on a field based on whether a checkbox is ticked or not.</p>
<p>Second rant over. Time to work some more.
</p>Marcelhttp://www.blogger.com/profile/13951354231483245521noreply@blogger.com0