Marcel Popescu

Taking a walk

2012-02-01T14:34:00.000+02:00

[Inspired by http://lifeistheteacher.wordpress.com/2012/01/07/my-only-experience-in-a-public-high-school-and-a-call-to-action/]

Frank was walking around, window-shopping while waiting for the time to come to perform at the near middle school. He was a sixteen-year old guitarist who was visiting a friend and got an offer to perform on a Friday in the friend's class. Frank was homeschooled so classes bored him to tears; he preferred to discover what was interesting in this part of the town instead of listening to a teacher drone about a subject he had absolutely no interest in.

Unfortunately, close to the time he was actually preparing to go back to the school, he got stopped by two cops who started asking questions about who he was and what he was doing there. It was clear they believed him to be a truant student but even so, Frank thought, why was it their business?

"I'm a musician; I'm performing at the school here in about half an hour."

"Right, kid, pull the other one," one of the cops said.

Frank was surprised at the disrespect. "Why do you assume I'm lying? And why is it your business anyway?"

"Kid, if I want your opinion I'll give it to you. Where do you live?"

Frank narrowed his eyes. He was starting to get pissed off. "I don't like how you guys behave. Is this your understanding of ‘protect and serve'? I'm a citizen minding my own business; please mind your own." Frank was not going to start anything, but if they annoyed him too much he was going to defend himself.

"Kid," the second cop said, "I think you need some time in jail to cool down. Put your hands behind your head and turn around."

Frank ignored his order. "So you think that might makes right, huh? You can force me to do what you want and that makes it ok? If I could kill you, would that be ok too?"

Both cops put their hands on their guns. "Is that a threat?"

Frank blinked. "Wow, you're stupid even for a cop. Yes, it's a threat."

"That's it!" one of them barked, pulling out his gun. "Hands behind your head, now; you're under arrest."

Frank ignored him once again. "Last chance; cool down and leave or this is the last time you're bullying anyone." He could feel the energy gathering around him and knew that his eyes would soon start to glow. From the descriptions of those who really knew him and what he could do, he knew that would look awesome and scary at the same time.

One of the cops grabbed his shoulder and tried to turn him around. Frank turned his head to look at him and touched him back. Everything went nuts as the cop's arm burst into flames.

The other cop had not pulled his gun before; he did now and shot at Frank. Frank switched his attention from the first cop, who was now screaming his throat raw, and started moving towards the second. The cop kept shooting and backing off, throwing his gun at Frank when the slide locked back. "What are you?" he screamed.

"A kid, I think you called me," Frank replied calmly, still moving towards him. "I just hate bullies. Have you learnt your lesson? Are you going to stop annoying peaceful people? On second thought," he continued without waiting for a reply, "I don't think I trust you." He pointed his finger at the cop who started screaming in fear, expecting to share his colleague's fate. "Might does not make right," he thundered, "but I'm not against using one's opinions against him. Don't make me come after you."

With that, Frank disappeared. A few seconds later, the fire that had by now engulfed the first cop completely went out and his skin started to heal. He was back to normal in less than a minute.

Conquest

2012-01-10T13:53:00.000+02:00

"Do not be afraid."

David jumped around, his body flush with adrenaline. He had been alone in the room, or at least that's what he thought. Fortunately, he had managed to put his pants on.

"Who are you?" he managed. Someone was there, in the room, on the other side of the bed from him. He couldn't understand how he got past the agents outside.

"My name is Ra'ken," the man said. "We need to talk."

"How did you get here?" David asked, trying to decide whether shouting would be a good idea.

"I will explain. Please finish dressing and then we can go outside so you can feel safer. I did not mean to scare you."David felt a bit ridiculous but he was old enough to know that just because someone tells you they don't want to harm you it doesn't mean it's true. He finished putting a shirt on. "Shall we?" he asked, turning his head back to the other man.

"After you," the man said.

David opened the door. The three agents stopped talking and turned their heads to him. "POTUS is up," one of them announced, letting everyone in the White House know that the day had started.

Five seconds later, all hell broke loose. David had made it to the middle of the room when Ra'ken followed him. As soon as one of the agents saw him he shouted "Stop! Identify yourself!" and pulled his gun. Ra'ken stopped and lifted his hands a bit.

"Wait," David said. He took his first good look at Ra'ken. The man, though he was starting to feel like that wasn't the case, was very tall - over seven feet. He had a long face and long, blond hair. "Are you an elf?" David asked before he could control himself.

"I have been called that, yes," Ra'ken answered. "I mean you no harm," he continued, looking at everyone in the room.

"Ok, first things first," David said. "How did you get here? In my room?"

"Magic," Ra'ken answered.

"Please be serious," David protested.

Ra'ken lifted his right hand with the palm up. A small globe of fire formed over it.David felt for the chair behind him and sat down. "What the hell?" he managed. The globe was still burning. "Are you seeing that too?" he asked the agents, who nodded back. He had to admire them, they seemed unmoved by the scene - all of them were still pointing their guns at the intruder.

Ra'ken closed his hand and the globe disappeared. "We need your help," he said.

The tension in the room was palpable. The president had cancelled everything on the day's agenda for his top advisers and set up an emergency meeting for 2 pm. The rumors were flying around, though nobody took the idea of a war seriously - it wouldn't have made sense for the president to know about it before the CIA or the NSA. Still, Joe thought, something big must be at stake for David to cancel everything; he knew how important the meet-and-greet sessions were for a new cabinet.

"Joe," asked Carol, the National Security Adviser, "do you know what this is about?"

"I'm just as much in the dark as you are," Joe replied, "and let me tell you, I don't like it a bit." Joe was David's closest friend since college and as soon as David got elected he named Joe as the Director of the CIA. It wasn't just a matter of whom you knew, it was too important a position for David not to give it to someone he knew he could trust.

At exactly 2 pm, David entered the room. "Please be seated," he said as everyone had raised from their chairs. After taking his place at the head of the table and waiting for them to settle down, he began. "Gentlemen, I'd like you to meet my guest, Ra'ken."

The same door David had used opened up and Ra'ken entered. Joe felt his jaw drop and by the reaction of those around him, he wasn't the only one in shock.

"Hello," the new guest said. "My name is Ra'ken; my species is called Erzhi but I have been called an elf before. Feel free to use either name."

"How?" Joe managed to ask.

David made a face. "That is indeed a problem. Ra'ken came here using magic. Before you start," he raised a hand to stop the exclamations of unbelief, "he can prove it."

"Indeed," confirmed Ra'ken. "For now, however, that is not important."

"Please have a seat," David said, pointing out to one of the available chairs, "and tell us why you came."

Ra'ken sat down and took a deep breath. "We are at war," he began.

"The race we are fighting is called the Moqqid. They are strong magic users and have a large empire - much larger than ours. They have at least twenty-seven planets that we know about."

Everyone around the table was stunned. Not one, but two alien races, one of them with a huge empire, and Earth found itself in the middle of a conflict.

Ra'ken continued. "We have fewer people and fewer resources than the Moqqid. We have been fighting for several decades and, even though we have won some battles, we are losing. The Moqqid have already captured three of our planets and we have no news about the fate of our people there. We are becoming desperate."

Joe started thinking - any conflicts the US was involved in would have to be cancelled right away. The contractors would not complain, given that the new war they were about to be involved in would be much larger in scale. If anything, they were going to be ecstatic. Just bringing back the troops from Asia would free over a hundred thousand people.

"We are going to need any technology you can give us," he interrupted. "Weapons would be best, but we can use anything - medical, logistics, anything. In exchange, we can probably supply ten thousand troops right away." Joe was looking forward to a heated negotiation.

Ra'ken turned his head to him. "I apologize for the misunderstanding," he said. "We have no technology. We also do not need your help to wage war. Humans are incapable of using magic; a single Moqqid warlock, even a low level one, would destroy all ten thousand human soldiers without being hurt."

David blinked. "What do you need from us, then?"

Ra'ken turned back. "Slaves," he answered.

Don't expose class internals

2011-10-16T20:06:00.000+03:00

I'm going to disagree a bit with Robert Martin, the author of Clean Code. In his G14 "Feature Envy" smell he uses the example of an method on an HourlyPayCalculator class that takes an HourlyEmployee argument and then uses its properties to do its job. That makes the method "envy" the HourlyEmployee class - the method "wishes it was inside the HourlyEmployee class".

So far, so good. Unfortunately, Robert continues with a counter-example to the feature envy smell; he uses the following example (Java code):


  public class HourlyEmployeeReport {
    private HourlyEmployee employee;

    public HourlyEmployeeReport(HourlyEmployee e) {
      this.employee = e;
   }

    String reportHours() {
      return String.format("Name: %s\tHours: %d.%1d\n",
      employee.getName(),
      employee.getTenthsWorked()/10,
      employee.getTenthsWorked()%10);
    }
  }

Robert says: "Clearly, the reportHours method envies the HourlyEmployee class. On the other hand, we don't want HourlyEmployee to have to know about the format of the report. Moving that format string into the HourlyEmployee class would violate several principles of object oriented design."

I believe there's a way to solve both problems. The HourlyEmployee class has to know about the concept of reporting; that's why we're exposing the getName() method. The only decision is whether we hide this concept or make it explicit. Here's a way we could make it explicit (I'll use C# because, while I can read Java, I'm not confident enough to write it):


  public interface HourlyEmployeeReporting {
    void Report(string name, double tenthsWorked);
  }

  public class HourlyEmployee {
    private string name;
    private double tenthsWorked;
    //... other stuff, but keep the internals hidden

    public void Report(HourlyEmployeeReporting reporter) {
      reporter.Report(name, tenthsWorked);
    }
  }

  public class HourlyEmployeeReporter: HourlyEmployeeReporting {
    void Report(string name, double tenthsWorked) {
      var report = string.format("Name: {0}\tHours: {1}.{2}\n",
        name, tenthsWorked / 10, tenthsWorked % 10);
      // do something with the report value, like writing it to a Logger
    }
  }

This way we haven't sacrificed any of the object oriented principles: everything is encapsulated, we can still have reporting and, quite importantly, reporting is exposed as a concept instead of being hidden just because we can use the internals of the HourlyEmployee class to do it.

[Note: I am not saying that exposing getName() doesn't work. I am saying that the way I'm showing is more object oriented.]

The Awakening

2011-10-06T12:42:00.002+03:00

The source of their Powers was never discovered; what nobody had doubted, however, was that there was a will behind it. Both the fact that there were exactly nine of them – one for each billion people – and the fact that each of them gained their powers on January 1st, 2050, at exactly midnight local time, made coincidence a too unlikely explanation.

Once it found out about their existence, the world called them superheroes; but amongst themselves they would use the name Champions.

The people would not discover the existence of the Champions for more than a year, though, because at the same time they had gained their powers, the Universe had started responding to spells. Some would call them wishes, or prayers; whatever the name, people's intentions were starting to directly affect the real world. Like many religions had insisted, it would only happen if the person making the wish genuinely believed in it; unlike what those religions had taught, the effect of the wish appeared to be completely unrelated to its content. Someone would pray for a million dollars, truly believing in the power of prayer; a week-long drought would be the result.

Very soon, in a matter of days, people realized what happened and started experimenting. Whatever the reason for this change, it appeared to be almost mechanical in the way it interpreted the spells. They had to be voiced - mere thinking did not seem to have an effect - but the language would appear to have little relevance. Wishing for a million dollars in English, French or Chinese would all cause a week-long drought, with maybe minutes in difference; on the other hand, wishing for a million Euros or Yuan would change the length of the drought significantly and work was in progress to determine the exact relationship. Intonation, number of people making the same wish, number of repetitions - everything had an effect.

It didn't take long before the experiments were formalized; by next Fall, Universities throughout the world began adding to their curricula courses in the new science of Magic. The governments were even faster in regulating the new Spellcaster profession. The American DHS was the first to have a Magic Crimes Division, but other agencies and governments followed suit very quickly. Even the Vatican announced that they were going to convene a special council to analyze the new situation.

Ideas for further development:

The DHS asks for the power to record everything, including private residencies, to discover people who make unlicensed wishes. It is granted, as long as the search is never done manually, only through an AI that can check warrants. The recordings are also supposed to be erased after an year but, with cryptographically-secure storage being practically free, this is not something you can count on.
A black market in spells develops very quickly.
The copyright lawsuits in spells dwarf anything previously seen with movies, music or patents; the next richest guy on the planet is a spell writer.
Religious fanaticism leads to armed confrontation against the government; even regular people are uneasy with the idea of the government regulating religious prayers.

TDD by example - first book

2011-10-02T22:11:00.000+03:00

This is an alpha version of a TDD booklet I wrote. As I said earlier, I plan on writing several of these.

TDD by example - Evaluating an expression [pdf]

It's a hands-on book; if you don't plan to follow it by writing code, it might not be of any help. The intended target are people who have done no TDD at all, or who tried it and couldn't make it work. If you have experience doing TDD, this book will probably look childish.

Feedback will be strongly appreciated. Positive if at all possible, but even "it sucks" is better than nothing :)

The book is also available from Amazon for $4.99 for Kindle (if you're in the US or Canada; I know Amazon changes the price in other countries) or $14.99 in paperback.

Science vs dogma

2011-09-29T00:44:00.002+03:00

Two hypotheticals:

1. You (common, everyday man) observe something occurring in nature. Every time it happens, you figure out that a specific something causes it. You emit the hypothesis that "A causes B" and devise a number of experiments to disprove it. You fail, and as far as you know everyone else also fails to disprove your theory. You tentatively accept the theory.

2. You observe something occurring in nature. It violates accepted dogma. You note that there is no known case where the alleged cause is known to actually produce the observed effect and, in fact, there is no actual proof that the cause even exists. You are told that you lack the inner grace that allows the high priests to verify that the cause does indeed exist; that there are secret rituals you're not privy to that they have used to confirm the truth of the dogma, and you're better off just accepting it as fact.

The first paragraph describes, for example, the idea that complex information is overwhelmingly, if not exclusively, generated by intelligent causes. A lot of people consider it to be a religious belief.

The second paragraph describes stuff like dark matter, string theory, ~~quantum physics~~, black holes and pretty much everything physicists declare to be scientific truth. (I love that phrase... "scientific truth". It has a gnostic ring to it.)

Edit: To clarify, by "quantum physics" above I meant the more esoteric interpretations, like the multi-verse theory; stuff that is not even verifiable, let alone verified.

Edit 2: This is a great example of a scientist understanding the science / dogma problem : The Van Allen Belts and Travel to the Moon.

Thou shalt not steal

2011-09-28T09:47:00.002+03:00

Jimmy was quite good at his craft, and that made him proud. His Pa, though – his Pa could never find out; he was a big stickler for that “honest work” bull, his Pa was. Jimmy was big for his fourteen years, so his Pa had started to bug him more often lately – “it’s time to start earning your keep, Jimmy”, “come work with me at the farm, Jimmy, we could use someone like you”. It was very annoying, even more so because Jimmy did want his Pa to be proud of him, but he wanted to get there his own way.

No, his choice of craft was not that of a farmer, or a carpenter, or anything like that. Jimmy was a pickpocket – a good one, if he said so himself. Minor stuff for now, but he had never been caught – the very thought of what his Pa would do to him if he found out about it gave him shivers – and he was hoping for a big score. Jimmy was sure that his big score was close – didn’t his Ma tell him that “good things come to those who wait”? So Jimmy waited, and practiced.

His Pa was also annoying Jimmy about his friends. He didn’t like them – he said they were a bad influence on him. That was not right – he had learnt his craft from them, and because of that extra money he could afford a few more trinkets. He was even less of a burden in the house expenses, so that had to be a good thing, no?

Still, that big score was the only thing that would solve their problems. His Pa worked at least ten hard hours a day, and barely got enough money to feed them all – Jimmy had a little sister that he loved, and he would bring her little gifts bought with his “earnings”. Sometimes even sweets, though that was really rare. If Jimmy could manage to score big, his Pa could work a bit less, his Ma could have enough money to buy something for herself, and his sister would have everything she wanted. They would all be so proud of him, and his Pa would have to admit he was wrong about Jimmy.

There was one problem that made that big hit unlikely, though, and Jimmy was aware of it: the economy was really bad. People were getting laid off left and right, some of them were even killing themselves when the banks got their homes. It looked like nobody had any money, so where could Jimmy find a sucker with lots of money? He thought about robbing a bank, but he didn’t know how to use a gun, and he wasn’t a bank robber anyway – he was a pickpocket, and he wanted to stick to what he knew.

Until one day… one of his friends found out that someone was going to carry a lot of cash on himself. Jimmy didn’t know how he knew that, and he didn’t care much. Since Jimmy was the best, he was recruited for the job. The sucker was old and careless – Jimmy could spot the big bundle under his jacket from a mile away. The guy was practically begging to be robbed, so what could Jimmy do but be a good fellow and help him out? It’s not like he was harming anyone – the guy surely had plenty more money where that came from, and anyone carrying a large bag of cash should hire some security to help, no? The guy must be a cheapskate on top of everything else.

The job was so easy a kid could have done it. Someone with even less experience than himself, Jimmy thought. The old guy was really bad at taking care of his stuff – Jimmy was definitely going to be a better master for the money than him. All it had taken was a few moments of carelessness, a well-timed distraction provided by one of Jimmy’s friends, and his dream was accomplished – the big hit, finally. Sure, the old guy was a little roughed up, but it wasn’t anything permanent – he was going to be fine. A rich guy like him was sure to have expensive doctors anyway.

Jimmy was ecstatic. Here was his chance to prove his parents wrong – especially his Pa. He was a man now, who would help his family. Of course, he had to take some time to celebrate the hit, and he had to split the loot with the rest of the gang. Jimmy kinda lost track of the celebration – he never had drunk so much ale before in his life, and that was a problem. Still, after the party ended, he still had enough to help his family for a good while, and Jimmy was nothing if not optimistic – more jobs like this would surely come.

So, two mornings after his hit, after trying to put back some order in his looks – it wouldn’t do to have his father start berating him as soon as he entered the house – Jimmy went back home with a smile on his face. His pockets were full of sweets for his sister Lexie, and he even had one good cigar for his Pa. He was going to give it to him and watch him smoke, and hear him tell that he was proud of Jimmy, and that he was indeed becoming a man.

As soon as he entered the house, Jimmy knew something was wrong. Both his Ma and Lexie were crying, and there were a few other people in the house. Some of the neighbors were there, and even Aunt Janet from Boston, who almost never came to visit.

“Ma?” asked Jimmy, “what happened? Sis?”

His Ma lifted her eyes and looked at him. “Oh, Jimmy,” she cried, “thank God you’re ok! I thought I had lost you too!”

“Ma? What do you mean? I’m ok, what happened? Where’s Pa? Lexie, don’t cry,” he continued, “here, I have some sweets for you, I know you like sweets.” He tried to give Lexie a handful of sweets, but she kept crying and didn’t even look at him.

“Oh, Jimmy, my poor boy. I... I…” his mother started crying again.

“Sit down, Jimmy,” said Aunt Janet. “Here, take a seat. Have some water to drink.” She pushed a glass of water to him.

“Auntie? What happened? Where’s Pa? Why is everyone crying?” Jimmy asked, really scared by now.

“Boy, I don’t know how to tell you this, so I’m just gonna say it,” said Aunt Janet. “Your father is dead. It seems that someone beat the farm administrator and stole the money he was taking to the bank to pay the mortgage, so the owner had to close the farm and sell everything. Your Pa couldn’t take the shame of having lost his job, so he hanged himself.”

“What?” Jimmy was sure this was a joke, a bad one. His vision was starting to darken, and he was making an effort to hear Aunt Janet, who kept talking.

“He left you a note,” Janet said. “He wanted you to know that he was always proud of you, and he knew you were going to become a great man. He said he was sorry for failing as a father, and he couldn’t bear to see the disappointment in your eyes. He said he was hoping you will do better than he did, and he asks that you forgive him, and that you take care of your mother and sister.”

TDD by example

2011-09-22T15:32:00.001+03:00

I intend to start a series of short "booklets", for lack of a better term, on the same idea of Growing Object-Oriented Software, Guided by Tests - show how a test-driven design process works for writing an application, from beginning to end. I realize I have hardly any readers, but is anybody interested in something like that? The articles will be free on my blog, of course, but I intend to also make them available for sale on Amazon.

I thought of stuff like writing an expression evaluator - 2 + 3 * 5 / (1 - 7) - and a postfix expression evaluator - 7 2 3 + * - but I tend to jump to math problems. On one hand, I need relatively simple problems, as I want to emphasize the TDD part; on the other hand, I don't want to write the next Stack implementation. Can anybody suggest topics?

Edited: follow-up at http://mdpopescu.blogspot.com/2011/10/tdd-by-example-first-book.html.

Flash fiction

2011-09-21T17:20:00.000+03:00

My house stands at the edge of the Earth.

It didn’t use to. Earth was a normal planet, a planet-sized planet as it were. Round. Not as big as Jupiter, of course, but not so small as to see its edges. I’ve been to cities that were bigger than what Earth looks like now.

There are only a few thousands of us left. We can’t say we’re alive, really – we’re almost zombies. Everybody that’s still here lost family and friends. We’re in the acceptance stage now… there was fighting, and incredulity, and mass suicides, but everybody seems to be resigned to whatever fate we have. Nobody has a clue what happened. It doesn’t even make any sense – no celestial body this small should have an atmosphere, and yet the air is perfectly breathable. A lot more breathable than it used to be, in fact – there’s no industry left on Earth, nothing to pollute. I think all the animals are gone too, except for a few pets. Oh, and lots of insects. The scientists were right; the cockroaches will inherit the Earth. Whatever will remain of it, if anything.

I don’t know what happened. Nobody does for sure. Maybe God was fed up with the humans again, and thought floods were passé? Maybe some evil wizard played with something that escaped his control? Whatever it was, it destroyed us all. The speed of the shrinkage has slowed down, but even if it were to stop completely there are too few people left now and too little space to live in. Some people have called this the Earth Spaceship; it finally became true.

Why am I even writing this? I have no idea. Maybe I’m an incorrigible optimist, even in the face of certain disaster. Maybe all the pessimists died out and I feel responsible for humanity. Maybe this is all a nightmare. Oh God, how I hope this is a nightmare! Somehow, though, I can’t seem to wake up from it.

At least we avoided global warming.

2011-09-17T23:20:00.001+03:00

“The fact is that the average man’s love of liberty is nine-tenths imaginary, exactly like his love of sense, justice and truth. He is not actually happy when free; he is uncomfortable, a bit alarmed, and intolerably lonely. Liberty is not a thing for the great masses of men. It is the exclusive possession of a small and disreputable minority, like knowledge, courage and honor. It takes a special sort of man to understand and enjoy liberty–and he is usually an outlaw in democratic societies.”
— H.L. Mencken, Baltimore Evening Sun, Feb. 12, 1923

Johnny Got His Gun

2011-09-17T17:08:00.001+03:00

I don't believe in Hell. Several years ago I decided to fully accept the preterist theory (short version: everything in the Bible has already happened); this has led me to universalism - everyone goes to heaven. I've held this belief for quite a while now.

I just changed it - somewhat. I suddenly believe in Hell, a place of such misery and torture that people can't really imagine it. A horrifying place. I also believe I'm in it right now, I just found a tiny bit which is somewhat less horrible. (I try not to make too many waves so I don't get moved to a more horrible part.)

This movie is why I think so - Johnny Got His Gun. It made me realize that politicians and soldiers are, if not the Devil, then something very close to it. They do their best to make other people's lives and homes a Hell. Hiroshima and Nagasaki might have been the clearest example, but read this description of the carpet bombing of Tokyo from Wikipedia:

On March 9 and 10 1945, B-29 Superfortresses were directed to bomb the most heavily populated civilian sectors of Tokyo. In just 2 days of bombing, over 100,000 of the population had burned to death from a heavy bombardment of incendiary bombs. Another 100,000 were left homeless.

We're in Hell. I now understand why some people commit suicide - they just discovered this, and can't handle it.

As for the rest of us - it's our job to make it better... though it's hard to be optimistic when I see how many people around me still worship the Devil.

Edit: For another horrible example, North Korea's death camps.

On science, peer review and consensus

2011-09-07T10:56:00.002+03:00

I find myself searching for these two articles a lot, so I thought I'd save them here. I can't recommend them enough.

http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/

http://s8int.com/crichton.html

Random N out of M

2011-08-20T15:18:00.011+03:00

Someone asked about randomly picking N elements out of an array of M, and I remembered reading about one way of doing that, so here it is:

    private static IEnumerable<int> Generate(Random rnd, int n, int m)
    {
      var i = 0;
      while (n > 0)
      {
        var r = rnd.NextDouble();
        if (r < (double) n / m)
        {
          yield return i;
          n--;
        }

        m--;
        i++;
      }
    }

The above C# method assumes 0 < N <= M and will return an IEnumerable<int> containing exactly N numbers in the 0 .. M - 1 interval. This method can even be used to pick items out of a stream without waiting for the whole stream to be produced first.

A simpler method would be to just do this:

    for (var i = 0; i < n; i++)
    {
      do {
        var next = rnd.Next(m);
      while (!alreadyGenerated(next));
      yield return next;
    }

but this one requires 1) saving all numbers (to check if one was already generated) and 2) having the actual data in an array instead of a stream (since the indexes are not generated in ascending order).

Edit: Per the observation from the first comment, the first fragment will never loop more than M times - an O(M) algorithm. The second fragment risks an O(M^2) if N is close to M.

Heinlein's rules of writing

2011-06-17T16:19:00.001+03:00

(H/T to Dean Wesley Smith)

1) You must write.
2) You must finish what you write.
3) You must not rewrite unless to editorial demand.
4) You must mail your work to someone who can buy it.
5) You must keep the work in the mail until someone buys it.

Story

2011-06-12T21:32:00.004+03:00

I wrote this story a few days ago. I have no idea what to do with it now... so here's the first step: put it on the blog.

If anybody gets here, by some amazing coincidence, please leave feedback even if it's only "it sucks" :)

The paradox maker

Jack ripped the envelope. “Dear Mr. Harroway,” it began, “we are happy to inform you that your application for a study of the local effects of the 2030 Tehran terrorist attacks has been accepted.”

“Yes!” he shouted. “Yes, yes, yes!” His work at cultivating important officials had finally paid off. The bribes didn’t hurt either.

Jack was a brilliant physicist. Almost forty-five years old – his birthday just last month – he had everything a scientist could want: a great career, the admiration of his peers, even a great family. He did have one obsession though: he wanted to go back in time and kill himself.

Why? He probably couldn’t answer, at least not without a great deal of thought, but it was mostly because he had been fascinated by the idea the first time he encountered it, in some long-forgotten book on paradoxes. It was just a “thing” – some people collected stamps, some women, he wanted a clear-cut paradox. He didn’t trust the grandfather version, because there was always the risk that someone else was his real father; he wanted to be absolutely certain that he had a genuine paradox. A science-fiction story he had read long ago claimed that the Universe rejected paradoxes and would actively fight against them. Good. Jack was up to it. Man versus the Universe, what could possibly be more awesome?

Getting here had taken a lot of work. Time travel was a heavily regulated business, and time travel in the close past was almost never granted. The politicians were too afraid of someone changing the result of the last election. Jack soon realized that he couldn’t come up with any plausible excuse for his trip, so he focused on solving the problem through the most direct means: bribery and old-boys networking. Of course, that meant cultivating important city officials, but academia was good for that. There were fund-raising events, and alumni who had the necessary connections. As time passed, he had become more and more obsessed with the idea, sacrificing everything to it.

His wife was his unknowing accomplice. High society parties were her idea of heaven, and the young enthusiastic idealist who had married a poor scientist was long gone. She had grown more sophisticated – and, he would have to admit, more elegant – raising to the occasion as a graceful hostess. They didn’t have any children, something they were both happy about, so she would fully focus on their social advancement. It was a marriage made in heaven… well, it would have been, if not for his hidden goal.

Well, this was it. His had all the approvals he needed to go back in time 20 years. That was normally something historians would do, but he claimed he intended to write a book on the subject, and that coupled with a heavy bribe was a good enough excuse. He was absolutely forbidden from interfering or warning anyone, of course – and, in fact, had absolutely no intention of doing that. He was implanted with a full immersion recording device which was supposed to serve as evidence of his being a good boy and doing exactly what he was supposed to do – research only, no actions that would change the timeline. He was fine with that – in fact, a recording of his accomplishing a genuine paradox was a bonus.

Two minutes till the deadline. Jack went back over the plan once more. He would have been 15 at the time, living in the same city and already intrigued by the idea of time-travel paradoxes. Time-travel had started being practical not long before, in 2025, but it was still mostly used only by the US government. The foreign governments were making a lot of noise about it, at the same time sinking huge amounts in trying to reproduce – or steal – the invention; in fact, the Tehran attacks were supposed to be related to that. Time travel was in the news all over the world – and young Jack Harroway, 15 years old prodigy student, was completely fascinated by it. It had ultimately led to him becoming a physicist… and to this day.

Five more seconds… four… three… two… one… flash. There was no reason for any effects to occur when someone moved through time, but most people reported some sensation – flashes, blackouts, short-term dizziness. He was here, close to his old home – he chose that spot beforehand, with nobody questioning it. This was it. The high he was on was incredible – adrenalin was pumping through his body, giving him a sensation of invincibility. He was going to create a real paradox.

First things first: confirm arrival. No accident has ever been reported, but his experiment was too important to trust others. Trust, but verify, wasn’t there a president who said that? If something wrong did happen, he had an emergency return button he could use, but he was worried more about failing his attempt than about being stuck in the wrong time or place. In any case, that was quickly solved; he put a quarter in a nearby newspaper box and verified that this was indeed June 12, 2030. The attacks would occur at 9pm EST, and he was supposed to have arrived at 4pm. The Sun was still pretty high on the sky, so he assumed the time was accurate. Satisfied, he looked around and started walking to his destiny. That sounded so important – his destiny. He almost felt like shouting to the people on the street – the attacks are nothing, the economy is nothing: you’re witnessing destiny in the making! Eh, what did they know?

There – the house was just as he remembered it. His father would still be alive – he had died of cancer when Jack was 38 – and his younger version would probably be playing some online RPG. He was quite a fan in his youth; he didn’t much like the fantasy RPGs, but the sci-fi-themed ones was something else altogether. What could be cooler than having your own spaceship and travelling among the stars, fighting pirates and rescuing civilians? Wait – creating a genuine paradox and fighting the Universe would surely qualify.

He rang the bell. After a few seconds, his mother opened the door. “Yes?” she said.

“Hello, is Jack home? I’m his physics professor.”

“Oh,” she said, flustered. “Is there anything wrong? Has something happened?”

“No, no,” said Jack. “This is not school related. We’re both playing the same online game, we’re in the same clan actually, and I wanted to discuss something with him.”

“Oh,” she replied, a bit colder. He remembered that – his parents were never supportive of his infatuation with online games. He couldn’t object too much, really, since he now pretty much agreed with them that they were a waste of time, but back then it had been a problem.

“Please take a seat,” his mother said. “I’m Melanie, Jack’s mother; just a moment until I get him here. He’s probably playing right now.”

“Nice to meet you, Melanie; I’m Robert,” said Jack.

“Nice to meet you, Robert; it will only be a second.”

The young Jack appeared shortly. “Yes?” he asked, not recognizing the stranger.

“Hi, Jack, “ the future Jack said. “Sorry to bother you at home… there is a problem with the Paradox Engine and we need to solve it before the next clan fight.”
Young Jack’s eyes became larger. “The Paradox Engine” was his idea of a password to recognize his older self; in fact, he only came up with it two days before. His breath quickened. He suddenly realized what this meant.

“Oh, yes, please come to my room to discuss it,” he said, turning back without waiting for a reply. “Mom, we’ll be in my room, please don’t disturb us,” he continued.

“Ok, honey. Professor Robert, would you like something to drink? A coffee?” she asked.

“No thanks, ma’am, I’m fine. I won’t be long, please don’t bother on my account,” Jack replied. He quickly followed his younger self to his room.

Once there, the young version closed the door carefully and then sat down. “So… we did it. Well, you did. You came back in time.”

“Yeah,” future Jack replied. “It took some doing, as you can see, but we’re finally here.”

“Oh, wow. I can’t believe it. I mean, I only came up with the password two days ago. In fact, I don’t think I read about paradoxes more than six months ago.”

“Yes, I know,” said future Jack. “That’s why I chose this day; I wanted to make sure everything is still fresh in your mind. So, are you ok with it?”

“Now?”

“I’d prefer it. I’m worried that something might happen to stop us. That story I read, about the Universe actively preventing paradoxes from happening, really did a number on me. The closer we get to it, the more worried I am.”

“I don’t think I remember that story,” said young Jack.

“Hmm… I don’t remember exactly when I read it, but I guess it was after your time.”

“Ok. Let’s do it. I’m as ready as I’ll ever be.”

“Do you want to close your eyes?”

“Ok. You’ll make it quick, right?” asked young Jack.

“Of course. You’re me, remember? I have no intention of causing myself any unnecessary pain.”

Taking a deep breath, future Jack removed a ceramic knife from his pocket. A metal one could have been detected by the time travel office and he didn’t want that to happen. Folding it open, Jack steeled himself and with a sudden move, plunged the knife into his younger version’s throat.

Young Jack’s eyes flew open and he tried to put his hands on his neck. He tried to say something, but old Jack pulled the knife out and hit a few more times, finally managing to hit the heart. Killing someone is difficult, he thought, even when they’re cooperating. He was quite nauseated – he was not a psychopath, at least he didn’t think so, and seeing a young man die, and blood everywhere, was having an effect. The smell wasn’t helping either; he felt very dizzy… the room started spinning… everything was splashed with red blood… or was it black?

Jack fainted.

When he came out of it, Jack was back in his home… just before he had left for the local branch of the Federal Bureau of Time Travel. He was worried that he dreamt the whole thing, but he could still smell the blood, and in fact his clothes and face were still dripping red. Puzzled, he went to take a shower and change his clothes, hoping that his wife wasn’t home. This doesn’t make any sense, he thought.

“Jack?” he heard his mother calling. “What?” he replied, before realizing that this was impossible. He didn’t live with his mother anymore – she was back in the old house. Looking around him and seeing nobody, he decided that he must have imagined the voice. It didn’t last long.

“Jack, dear, I’m going out to buy some food. If your father comes home before me, tell him I won’t be long.”

Suddenly, he was back in the old room. Looking down at his hands, he realized he was in his 15-years old body. What was going on here?!

Over the next few days, he switched back and forth between his old and new selves. He had no control over the change, and he was starting to worry about getting out of the house – if the switch caught him while crossing the street, he had no idea what might happen. Unfortunately, that was not really a problem – causality seemed to have lost its power as far as he was concerned. Impossible things started happening – he would pour water into a glass, only to find it still empty, strong winds would start inside his room, he would hear or see things that nobody else did… worse of all, he started getting seizures.

He tried asking for help, explaining what he did to other physicists. Nobody believed him, mainly because there was no record of him ever going back in time, but also because his frequent body switches made the whole explanation incoherent. Jack tried going to a psychiatrist, but the medication did nothing but make him even dizzier. His apparent mental health worsened at an incredible speed, so in a couple of weeks his wife had no choice but commit him to an asylum.

Jack is happy now. All the inmates know that he’s the sole paradox maker in the Universe.

Communist USA

2011-06-12T00:29:00.004+03:00

Here's something I never thought I'd see in the US (as bad as I believe it has become):

Ideas Matter, a pro-IP propaganda site build by Microsoft and others, has the following on their About page:

In the words of President Barack Obama: "Our single greatest asset is..."

Bear with me while I'm freaking out here. I grew up in communist Romania until I was 18. Any time you wanted to write something - including in school essays - we would have to quote the beloved president. Freaking programming books, few as they were, would have something that the president said in their introduction.

What the hell???

The pursuit of knowledge

2011-01-13T14:40:00.001+02:00

From http://freedomtwentyfive.wordpress.com/2011/01/09/why-im-leaving/

But, you ask, what about the glorious pursuit of knowledge? Won’t you be dedicating your life to advancing the state of human knowledge? If you’re asking that, I guess you’ve never actually been inside a Western University: They are cesspools of mediocrity and ideological cant. The contemporary university is quite possibly the worst place in the world to actually think freely and come up with new ideas. As a career academic, you will frequently have to choose between honesty and your career. And remember, if you aren’t willing to make the “right” choice, there are a thousand others lined up behind you who will.

Excel gotcha

2011-01-06T14:27:00.003+02:00

Just got bit by a... I can't call it "bug", but at least surprising behavior, in Excel 2007 (though I believe it is the same even in other versions).

I had a column of text (file names) in a csv document which was opened in Excel and re-saved as a regular Excel files. The recipient of that document asked me while some of the file names were in scientific notation :) It puzzled me for a bit, until I realized that a lot of the file names had the form 12E12345. When the part after the "E" was small enough (I didn't bother searching for the limit, but I'd make a wild guess and say below 300), Excel would interpret that string as a number in scientific notation - so 01E00212 became 1E+212.

The program I used to generate these csv documents knew to prefix strings made up of only digits and starting with zeroes with an apostrophe - like '00123. This one, however, caught me unprepared :)

Complexity and chance

2009-11-06T15:08:00.001+02:00

In one of the Miles Vorkosigan books, the main character has a device installed in his brain and a remote that can trigger a seizure - he needs that because otherwise he would get uncontrolled seizures, usually at moments of high stress (when that's the last thing he needs). One of his problems is having that remote triggered accidentally, so the doctors assure him they coded the connection - no random signal can trigger the seizure, only the remote.

What are the possible dangers here?

1) Accidental signals that would match the frequency of the brain device, thus generating a false positive.

How does one defend against this possibility? A small code of some sort is usually required for triggering the functionality - like the 4-digit code I have to enter when my Nokia cellphone locks the keys (to prevent me from accidentally dialing out when I keep the phone in my pocket).

2) Intentional signals from would-be hackers who want to trigger the device.

In this case, one would need a much more complicated mechanism - possibly a challenge-response sequence where the remote must confirm that it knows a secret. (The book does not explicitly say this but, with Miles formerly in ImpSec and currently one of the several Imperial Auditors, we can assume something like this was actually employed.)

Note the difference? Against randomness, a very simple (= of low complexity) code is sufficient. It is unlikely to the extreme that I would accidentally press my Nokia cellphone's 4-digit code while carrying it in my pocket. It is practically impossible that I would manage that with an 8-digit code. If that code is entered, the phone software can safely assume I *meant* for it to happen - chance simply cannot account for that complexity. (Again, that's a 4-digit code, which is around 14 bits of information - 2^13 = 8,192 < 10,000 < 2^14 = 16,384.)

However, intelligence CAN overcome quite a hefty complexity barrier. Millions of man-years have been spent on creating ciphers that are supposed to last against other millions of man-years working to defeat them. This is VERY STRONG evidence that, given a complex system, we can safely eliminate chance as the cause. While chance can overcome complexity barriers of a few bits - even of dozens of bits, given enough time and the right circumstances - once we get in the realm of hundreds of bits (the CSI barrier) it is absolutely ridiculous to insist on claiming it is a reasonable possibility.

So - what created life, again? :D

Maintainability

2009-08-01T15:14:00.002+03:00

Lots of noise on this subject - as far as I know, it started with this article by Patrick Smacchia that has apparently annoyed Ayende, who replied to it (and followed here). A few days later, Ayende replies to an article by Frans Bouma to explain what he means by maintainability - a definition that goes against what Patrick believes to be true.

Now that you're up to date, my opinion :)

An analogy from C. S. Lewis helps here. To claim that using toothpaste is good, he said, does not mean that if you find someone with bad teeth, but who uses toothpaste all the time, and someone who never used it, but has great teeth, you have disproved the statement. What matters is: given the same person, is he going to be better off by using toothpaste or not?

To apply the analogy: given the same person, whether an average developer from the street or Peter Norvig (or Ayende or Frans or Patrick), which application is he going to find more maintainable: one with - say - hundreds of cyclical references, or one layered as Patrick argues should be done?

In my opinion, this helps to nullify the inherent subjectivity of the issue: an application is more maintainable than another if, on average, more programmers would find it so. (I remember reading about someone who wrote code that would take into account the time it took to execute a piece of code and then read the next value from the disk — which was exactly what he needed at that point; he knew where the disk head would be given that precise timing. That was a maintainable application for him — less so for anyone else.)

Of course, my definition (if I can call it that) still depends on the idea that there are lots more competent programmers than Daily WTF candidates; I do not know if this is in fact the case :)

Monkeysphere?

2009-07-11T11:22:00.002+03:00

Interesting article at http://www.cracked.com/article_14990_what-monkeysphere.html. They didn't elaborate on the subject of religion, but I think (as some commented) that this is why the so-called golden rule is emphasized: trying to love everyone as you love yourself is another way of saying "realize they're also people just like yourself".

No big discovery, I realize :) but I liked the connection.

More on biblical laws

2009-07-10T18:33:00.003+03:00

Sort of a follow-up to my previous article - I just read "Why did the pig become so unpopular?" which added a lot of economic reasons for forbidding pig eating to the one I considered before (the disease part - pig meat has to be cooked really well to be safe, otherwise you risk a bunch of problems, which in Israel 5000 years ago is serious).

Interesting.

More speculation...

2009-06-23T09:49:00.002+03:00

... in the vein of this.

Jesus says in Mark 11:23 I tell you the truth. Anyone may say to this hill, "Go and jump into the sea." He must not doubt in his heart, but he must believe that he will have the things he asks for and he will have them.

What if Jesus didn't mean "if you have faith, I will move the hill (mountain) for you", but "if you have faith, the hill will move because the Universe was built to respond to human volition"? We know that the human body responds to human will sometimes (placebo effect); there's a speculative part of physics (quantum physics) that has as a basic tenet that human observation can influence the outcome of physical processes.

Why the need for "unwavering" faith, then? Well, the Universe is probably a simple mechanism, if you're not very sure of what you want it cannot respond to your will. Maybe there's a "faith threshold" that triggers the mechanism; or maybe there's a required ratio between the "amount" of faith and the desired effect (so less faith would be required for moving a pin than for moving a mountain).

As I said - speculation; it's not something I actually believe, for now it's just an amusing idea.

On extending DBMSes

2009-06-04T13:19:00.004+03:00

Problem: you have some data for your company. The data has to be accessed from multiple applications (think accounting, personnel, sales and so on). How do you make sure business rules are obeyed - because if there's a bug in even one application, it will ruin the data and thus every other application.

I agree with pretty much everything Fabian Pascal has to say about relational databases. The RDB *is* the place where data should live, and ideally it should be the place for business logic too - because business logic is about data.

However, there are a few problems with this - mainly because of a lack in implementation, not in the theory of the relational database:

No existing DBMS (Database Management System) - the actual implementation of the relational database theory - can support thousands of concurrent users, let alone millions.

It is hard to do user management / authentication in the same database as your business data - in fact, it might be a good idea to have separate databases for users (think Active Directory) and the actual data you care about.

The SQL language sucks for general purpose programming. It is difficult to write complex business rules in, and even more difficult to port to another vendor.

For these reasons, one should write an application whose main purpose is to "guard" the (main) database and enforce the business rules; think of it as "extending" the DBMS. [1] All other applications should go through this "gatekeeper" application.

I guess this is some sort of "pattern" - an architecture that can be used to solve a number of problems.

[1] Note: some DBMSes allow you to write "plugins" or "user functions" in a different language. The problem is, these are also not easily portable, so I would recommend against it.

WCF - inheriting from a common interface

2009-05-24T23:07:00.010+03:00

At work I have a few dozen (and will at some point get to hundreds) workflows with the same structure:

[XmlSerializerFormat]
[ServiceContract(Namespace = "http://schema.company.com/messages/")]
public interface IBasicContract<TRequest, TResponse>
where TRequest : class
where TResponse : class
{
[XmlSerializerFormat]
[OperationContract(Name = "GetReport",
Action = "http://schema.company.com/messages/GetReport",
ReplyAction = "http://schema.company.com/messages/GetReportResponse")]
[ServiceKnownType(typeof (Aggregate))]
TResponse GetReport(TRequest inquiry);
[XmlSerializerFormat]
[OperationContract(Name = "GetRawReport",
Action = "http://schema.company.com/messages/GetRawReport",
ReplyAction = "http://schema.company.com/messages/GetRawReportResponse")]
string GetRawReport(string guid);
[XmlSerializerFormat]
[OperationContract(Name = "GetArchiveReport",
Action = "http://schema.company.com/messages/GetArchiveReport",
ReplyAction = "http://schema.company.com/messages/GetArchiveReportResponse")]
[ServiceKnownType(typeof (Aggregate))]
TResponse GetArchiveReport(string guid);
}

I have long wanted to refactor all the workflows to use a common interface, instead of copy-pasting the same code all over the place. Today, after spending 3 days on this, I finally made it :) so I'm saving it here for reference.

This is the common implementation of the above interface:

[ServiceBehavior(Namespace = "http://schema.company.com/messages/")]
public abstract class BasicWorkflowSvc<TRequest, TResponse, TWorkflow> : IBasicContract<TRequest, TResponse>
where TRequest : class
where TResponse : class
where TWorkflow : class
{
public virtual TResponse GetReport(TRequest inquiry)
{
TResponse res = null;
using (var wr = new WorkflowRuntime())
{
var waitHandle = new AutoResetEvent(false);
wr.WorkflowCompleted +=
delegate(object sender, WorkflowCompletedEventArgs e)
{
res = (TResponse) e.OutputParameters["wfOutput"];
waitHandle.Set();
};
var arguments = new Dictionary<string, object>();
arguments.Add("wfInput", inquiry);
var wi = wr.CreateWorkflow(typeof (TWorkflow), arguments);
wi.Start();
waitHandle.WaitOne();
}
return res;
}
public virtual string GetRawReport(string guid)
{
using (var db = new DBAccess())
return db.LoadGUID(guid);
}
public abstract TResponse GetArchiveReport(string guid);
}

This is the client used to call the workflow:

public class BasicSvcClient<TRequest, TResponse> : ClientBase<IBasicContract<TRequest, TResponse>>, IBasicContract<TRequest, TResponse>
where TRequest : class
where TResponse : class
{
public BasicSvcClient()
{
}
public BasicSvcClient(string endpointConfigurationName) :
base(endpointConfigurationName)
{
}
public BasicSvcClient(string endpointConfigurationName, string remoteAddress) :
base(endpointConfigurationName, remoteAddress)
{
}
public BasicSvcClient(string endpointConfigurationName, EndpointAddress remoteAddress) :
base(endpointConfigurationName, remoteAddress)
{
}
public BasicSvcClient(Binding binding, EndpointAddress remoteAddress) :
base(binding, remoteAddress)
{
}
public TResponse GetReport(TRequest inquiry)
{
return Channel.GetReport(inquiry);
}
public string GetRawReport(string guid)
{
return Channel.GetRawReport(guid);
}
public TResponse GetArchiveReport(string guid)
{
return Channel.GetArchiveReport(guid);
}
}

This is how I call the workflow, using the address retrieved from

using (var wf = new BasicSvcClient<ProductRq_Type, ProductRs_Type>(
new BasicHttpBinding("httpsDataEndpoint"),
new EndpointAddress(tools.FindEndpointAddress("Product.IProductSvc"))))
{
txtConfirmation.Text = wf.GetReport(request).AsXML();
}

Finally, these are the helper functions (in the tools static class):

public static ChannelEndpointElement FindEndpoint(string endpointName)
{
var cf = (ClientSection) ConfigurationManager.GetSection("system.serviceModel/client");
foreach (ChannelEndpointElement endpoint in cf.Endpoints)
if (endpoint.Name == endpointName)
return endpoint;
return null;
}
public static string FindEndpointAddress(string endpointName)
{
var endpoint = FindEndpoint(endpointName);
return endpoint == null ? null : endpoint.Address.ToString();
}

Well... hope this helps anyone else who has this idea :)

Multi-threaded server

2009-05-22T00:42:00.003+03:00

This is a reply to this question on stackoverflow.

Many people "feel" that allocating a thread per socket is a bad idea. This might be true for 10,000 sockets, I don't know; however, for 300 threads which are idle most of the time, this is really not an issue - and hasn't been in Windows since at least Windows 2000.

Here's how this would look on my system:

I am going to put the code in the answer to the stackoverflow question; this is just a placeholder for the above image, mostly :)

Acting good without being good

2009-03-17T15:59:00.001+02:00

From this comment:

I seem to be known to many of those around me as an all around nice guy, a good guy even. But I'm an asshole, a self-absorbed prick. (...) I often find that I'm doing good things out of an awareness that I should want to do them, than because I actually feel that desire.

This could have been written by me. Ouch.

Single Responsibility Principle - a practical application

2009-02-10T13:40:00.002+02:00

Many times during the lifetime of a project I find myself writing something like this:

method A(){
...
var something = B();
...
}

method B(){
...
var somethingElse = C();
...
}

It doesn't take long to realize that this could be written in a different way:

method A(){
...
var somethingElse = C();
var something = B(somethingElse);
...
}

method B(somethingElse){
...
}

The problem is - which way should I prefer? Which way is "better"? I have struggled with this for a while now.

Today I just realized something: SRP (as applied to methods) says each method should only have one reason to change. (I was reading this article when I had this epiphany.) If something changes in C(), that effect will "ripple" to both B and A in the first case. In the second case, it is much more likely that it will only affect A. Which is fine - it is A's responsibility to "join together" the two other methods, so it's ok if it's affected by them. B, however, has other responsibilities and only needs a result from C() - so it should be isolated from it, as it were.

Somewhat abstract so far :) I will try to add a concrete example to this post the next time I hit this issue. Who knows, I might be wrong :P

I love Paul

2009-01-22T19:59:00.000+02:00

1 Cor 6:12

"Everything is permissible for me" — but not everything is beneficial. "Everything is permissible for me" — but I will not be mastered by anything.

I'm an idiot

2009-01-22T19:40:00.003+02:00

I worked on a crapload of small ASP.NET projects where I spent a lot of time on user management: add users, edit users, change roles, verify roles and so on. This includes tiny projects like a tiny website where the content of about 10 pages can be edited by the customer (mostly by adding news). They'll never have more than one person doing these edits, their primary business is totally unrelated to computers.

Well, I was reading this article by Scott Hanselman and what do I see:

Does it need an administrative console?, I ask. He says that'd be nice, but it wasn't spec'ed out. He figured this would be pretty low rent. The client even suggested (gasp) that they could just maintain a local XML file.

How on Earth can I be so stupid? I always knew that ASP.NET had this feature... it just never crossed my mind to use it. Dammit.

Yes to mentoring, no to certification

2009-01-22T18:59:00.003+02:00

Great article here. I hate the programmers dragging the certification bogey-man every time they have a problem with someone's code. Yeah, we really want to have the problems of the medical field. This article is a breath of fresh air.

Take that, ServiceLocator denigrators!

2009-01-21T23:29:00.002+02:00

From Inversion of Control Containers and the Dependency Injection pattern by Martin Fowler:

The choice between Service Locator and Dependency Injection is less important than the principle of separating service configuration from the use of services within an application.

Ha! :P

More on MVC

2009-01-21T23:05:00.007+02:00

Follow-up to this.

This is the structure I've settled on for the time being:

View

Controller

Service

Model

Repository

Data layer (eg, Linq to SQL)

Data storage (eg, SQL database)

The View, Controller and Service communicate through DTOs (aka ViewModels); the View and Controller never see the (domain) Models. I'm not 100% decided about the Service - Repository interface: should the Model objects be the same as the Data Layer objects? For purity reasons I'd say no - it's a bad idea to return Linq to Sql objects to the Service because it will cause problems when trying to switch to (say) Entity Framework. However, I haven't yet worked on a project big enough that the benefits of this separation outweigh the drawback of creating 3 different classes for the same entity (Linq Order, Model Order, DTO Order). I'll have to see what happens when I actually do work on a larger project (with this architecture I mean, I work on huge projects but not with this structure).

The Service has two types of methods:

Reads, where it asks the repository for a list of Models (eg Products).

Writes, where it asks the repository for a Model and then asks the Model to do something.

This way I avoid the anemic domain anti-pattern - the Service layer can stay thin and let the Models provide (most of) the business logic - querying is not completely a business logic concern. (It is more of a reporting concern instead of a transaction concern.)

Speaking of that, I also need to come up with a way of saying "commit" when I'm doing transactions; each Service method must be a transaction, but I still haven't come up with a way of doing that. (Maybe the EntityService base class - all my Services inherit from it - should have a Commit method that in turns calls for example DataContext.SubmitChanges(). However, that would link the Service to a specific Data Layer, bypassing the repository/ies, which I'd rather not do. Hmm.

I already clarified the structure of the repository (if only to myself, since I haven't posted my last changes to the interfaces and Linq implementation ). In theory, it shouldn't be a big deal (a day or two) to write a Data Layer for, say, SQLite.

Well. Good place to save my current thinking on the subject. Hopefully I'm not too far off-mark.

TDD and discipline

2009-01-14T19:01:00.003+02:00

In a comment on his article, Karl Seguin has this to say about the usefulness of TDD:

My best suggestion was to unit test as you go, and most of these problems would have ironed themselves out. They never would have gotten the 100+ line controller actions under test, and would have had to address the problem upfront.

Well... unfortunately, that only works for someone who truly believes in the value of TDD, figures out the "hint" - and then is disciplined enough to do the right thing. You know what my reaction is, in these cases? Screw the tests, I have code to write!

Which, of course, is the wrong thing to do. I feel the need for a serenity prayer...

Nice methods in the ServiceLocator class

2009-01-14T00:11:00.003+02:00

I just like the way these methods look. I'd take the ServiceLocator class to a job interview :)

/// <summary>
/// This is used by the application when it needs an instance of T
/// </summary>
/// <param name="T">Type that the application needs an instance of</param>
/// <returns>An instance of T</returns>
public static object GetInstanceOf(Type T)
{
lock (containerLock)
if (container.ContainsKey(T))
return container[T]();
return CreateInstance(T) ?? (OnTypeNotFound != null ? OnTypeNotFound(T) : null);
}
/// <summary>
/// Creates a new instance of T using the "best" constructor (the one with the biggest number of arguments that are known to the service locator, with the default constructor being the last attempt)
/// </summary>
/// <param name="T">Type to instantiate</param>
/// <returns>A new instance of T or null if no suitable constructor has been found</returns>
public static object CreateInstance(this Type T)
{
var constructor = GetBestConstructor(T);
return constructor != null ? constructor.Invoke() : Activator.CreateInstance(T);
}
private static ConstructorInfo GetBestConstructor(Type T)
{
return (from c in T.GetConstructors(BindingFlags.Public)
let parameters = c.GetParameters()
orderby parameters.Length descending
where parameters.All(p => Contains(p.ParameterType))
select c)
.FirstOrDefault();
}
private static object Invoke(this ConstructorInfo constructor)
{
var arguments = constructor
.GetParameters()
.Select(p => GetInstanceOf(p.ParameterType))
.ToArray();
return constructor.Invoke(arguments);
}

Really weird C# code

2009-01-13T12:01:00.007+02:00

My ServiceLocator (which has been further updated since my last post on it, so on the off chance that anybody wants the latest version, leave me a comment) can only register functions with no parameters - Func. What happens if one wants to register a function with one or more parameters?

As an example, let's say that I have many possible connection strings, and in some cases when I want an instance of DataContext I have to initialize it with one of those. Since my ServiceLocator doesn't accept functions with parameters, I cannot write

var dc = ServiceLocator.GetInstanceOf<DataContext>(connectionString);

Here's a way of transforming a function with parameters to one without: create a function that returns the original function. That is, given

Func<T1, T> f;

I need another function that will return f:

Func<Func<T1, T>> g = () => (a) => f(a);

So, in order to register my DataContext I am now going to use the following syntax:

Bind<Func<string, DataContext>>.To(() => (connectionString => new DBDataContext(connectionString)));

and to retrieve the instance and initialize it correctly, I must use:

var dc = ServiceLocator.GetInstanceOf<Func<string, DataContext>>()(connectionString);

Not extremely clear, so some helper functions (and confining in a library of sorts) are probably needed.

Edit: I just made a change to the ServiceLocator code; one can now write:

var dc = ServiceLocator.GetInstanceOf<string, DataContext>("connectionString");

This is the overload for one argument:

public static T GetInstanceOf<T1, T>(T1 arg1) where T : class
{
return GetInstanceOf<Func<T1, T>>()(arg1);
}

I only have 3 overloads (up to Func) but that should be enough for anything I might need in the foreseeable future. Oh, and I can still have / use the initial binding too (the one with no connection string). Cool :)

Be careful with sample code

2009-01-09T15:39:00.004+02:00

Found this code somewhere, it doesn't matter where:

public override bool ValidateUser(string strName, string strPassword)
{
//This is the custom function you need to write. It can do anything you want.
//The code below is just one example.
// strName is really an email address in this implementation
bool boolReturn = false;
// Here is your custom user object connecting to your custom membership database
MyUserProvider oUserProvider = new MyUserProvider();
MyUser oUser = oUserProvider.FindUser(strName);
if (oUser == null)
return boolReturn;
// Here is your custom validation of the user and password
boolReturn = oUser.ValidateUsersPasswordForLogon(strPassword);
return boolReturn;
}

The comments are there because it's a sample, not because they should be there in production code, so that's fine. My problem is with the boolReturn variable.

bool boolReturn = false;
// Here is your custom user object connecting to your custom membership database
MyUserProvider oUserProvider = new MyUserProvider();
MyUser oUser = oUserProvider.FindUser(strName);
if (oUser == null)
return boolReturn;

First of all, one rule (Code Complete should really be required reading for programmers) is that you want to keep assignments to a variable as close as possible to the place where the variable is used. The oUserProvider and oUser variables have nothing to do with the boolReturn, so let's switch the order:

// Here is your custom user object connecting to your custom membership database
MyUserProvider oUserProvider = new MyUserProvider();
MyUser oUser = oUserProvider.FindUser(strName);
bool boolReturn = false;
if (oUser == null)
return boolReturn;

Ok... is the boolReturn variable used anywhere after the return? Not before it's reassigned. That means that, for all intents and purposes, the scope of the variable boolReturn ends here, and a *new* boolReturn variable is created. (Think about it.)

Let's rewrite the code accordingly (think it through to verify that nothing was changed in the meaning of the code):

public override bool ValidateUser(string strName, string strPassword)
{
//This is the custom function you need to write. It can do anything you want.
//The code below is just one example.
// strName is really an email address in this implementation
// Here is your custom user object connecting to your custom membership database
MyUserProvider oUserProvider = new MyUserProvider();
MyUser oUser = oUserProvider.FindUser(strName);
bool boolReturn = false;
if (oUser == null)
return boolReturn;
// Here is your custom validation of the user and password
boolReturn2 = oUser.ValidateUsersPasswordForLogon(strPassword);
return boolReturn2;
}

Progress. This change makes visible another problem: why set a variable if you're only using it once? (Ok, it makes sense for complicated calculations, but it's not the case here.)

public override bool ValidateUser(string strName, string strPassword)
{
//This is the custom function you need to write. It can do anything you want.
//The code below is just one example.
// strName is really an email address in this implementation
// Here is your custom user object connecting to your custom membership database
MyUserProvider oUserProvider = new MyUserProvider();
MyUser oUser = oUserProvider.FindUser(strName);
if (oUser == null)
return false;
// Here is your custom validation of the user and password
return oUser.ValidateUsersPasswordForLogon(strPassword);
}

Frankly, I find the ?: operator to be a great help, so my code would tend to look like the next example, but I understand some people get confused by it, so take your pick:

public override bool ValidateUser(string strName, string strPassword)
{
//This is the custom function you need to write. It can do anything you want.
//The code below is just one example.
// strName is really an email address in this implementation
// Here is your custom user object connecting to your custom membership database
MyUserProvider oUserProvider = new MyUserProvider();
MyUser oUser = oUserProvider.FindUser(strName);
return oUser == null
? false
: oUser.ValidateUsersPasswordForLogon(strPassword);
}

Finally, what's up with the (bad) Hungarian notation? The fact that the name (or email, whatever) is a string or something else is relevant for the compiler, not for the programmer. The good Hungarian notation can be useful, but most people seem to only know about the bad variant. Let's get rid of it altogether:

public override bool ValidateUser(string name, string password)
{
var userProvider = new MyUserProvider();
var user = userProvider.FindUser(name);
return user == null
? false
: user.ValidateUsersPasswordForLogon(password);
}

To me, this looks much better than the original. Doesn't it?

MVC explained

2009-01-08T16:51:00.014+02:00

... to myself, as I'm trying to learn it :)

This is how I'm organizing a project I'm currently working on:

View - this is used to render the model as HTML, since this is an ASP.NET project. The View can contain view logic, like displaying negative numbers in red.

ViewModel - this is not necessarily the domain model (database table), but a view-specific model. An Order view, for example, needs to display data from an Order, multiple OrderDetails, a Customer and possibly a Seller (and maybe more). Right now, I handle this case by making a composite class with several properties:

public class OrderModel
{
public Order Order { get; set; }
public IEnumerable<OrderDetail> Details { get; set; }
public Customer Customer { get; set; }
public Address ShippingAddress { get; set; }
public bool CanEditShipping { get; set; }
}

This is basically a DataTransferObject and is used as such by the view. I realize it would be safer if I created an immutable struct with the necessary data and pass that to the view, but I am too lazy and don't have a tool to do that for me right now. I think this is what Microsoft tried to do with their ViewData dictionary, since it's (normally) strictly write-only for the controller and read-only for the view. This might be easier in C# 4.0, because I can discover properties at runtime, so I can write something like this:

return View(new { Order = order, OrderDetails = order.Details, Customer = customer, ShippingAddress = order.ShippingAddress, CanEditShipping = true });

and still use Model.CanEditShipping in the view. (It still requires some discipline to use Model.OrderDetails instead of Model.Order.Details, of course.)

Controller - this has three responsibilities in my application: to use the business logic to retrieve the domain models (representing records), combine those into the model as defined above and choose the right view to apply or the correct path to re-route to (this is the "controller logic"). The controller is very "skinny", usually a couple of lines per action.

Repository - this is the business logic.

Domain model - aka tables / records / plain C# objects. Can contain "model logic", like a GetMainImage() method on the Product object that can decide to "promote" an image as main if none of those associated with the product have been flagged as such.

Any glaring mistakes?

Edit: I just found this article by Jimmy Bogard

Edit (Jan 16, 2009): Oopsie. It seems that I'm going to have to revise this.

ServiceLocator anti-pattern?

2009-01-07T23:48:00.003+02:00

I was trying to reply to this article but for some reason I cannot leave a comment there... there's no error message, my comment is simply ignored. So I'm replying here, hopefully I'll be able at some point to at least reply with the address of this article.

Ok... so if I understand this correctly, Nick's problems with the ServiceLocator pattern are:

1) Because it's global, it cannot be tested
2) Sometimes we want to return the same instance for all Resolve() calls, sometimes a thread- or HttpContext-specific instance, sometimes a new instance each time; can't do that with a dictionary

(I'm ignoring the points in the "Problems with..." section because I can't understand any of them. "All of the robustness that is achieved by forcing components to publicly declare dependencies is lost." - why? "When dependencies are retrieved from a global container, the container does not know the identity of the requestor, and is forced to return the same implementation every time." - this is a GOOD thing. You want an instance of X, it shouldn't depend on the requestor - if it does, make many kinds of Xs - it's obviously not the same X. "Concurrency, re-entrancy, circular reference detection, and component lifetime management are much harder/messier in a global container based solution." - again, why?)

It's weird, a real coincidence, but a few days after he wrote this article (which I've only discovered half an hour ago) I wrote a ServiceLocator implementation. This was my first post on the subject, followed by this one and finally this. As shown in the last post, this is one way it can be used:

Bind<IProductRepository>
.To(() => new ProductRepository())
.CacheBy(InstanceScope.HttpContext);

Can you take a look at this and point out where I'm wrong? I agree, right now I do NOT use any kinds of magic to get rid of the

var repository = ServiceLocator.GetInstanceOf<IProductRepository>();

calls, but I do not actually find that a problem (at least not yet). In fact, if I could find a way that doesn't look forced to replace it with something else, I will probably look into it. (Decorating properties or constructors with [Inject] attributes looks a little weird, but maybe I'm mistaken.)

I've started using this in a project and, so far, everything seems to be OK. I'm definitely open to the idea that problems are lurking in the background though, so I'll appreciate any hint you can give me.

Returning a default value in case of errors

2009-01-07T15:46:00.004+02:00

Ever wrote something like this?

if (a != null && a.b != null && a.b.c != null)
result = a.b.c.d;
else
result = null;

I hate it. Given that I'm processing some XML files whose schemas I cannot change, there's not much to be done but bite the bullet.

This page gave me the idea for the following helper functions:

public static T Eval<T, ExceptionClass>(Func<T> func, Func<ExceptionClass, T> onError)
where ExceptionClass: Exception
{
try
{
return func();
}
catch (ExceptionClass e)
{
return onError(e);
}
}
public static T Eval<T>(Func<T> func, Func<Exception, T> onError)
{
return Eval<T, Exception>(func, onError);
}
public static T Eval<T>(Func<T> func, T defaultValue)
{
return Eval<T, Exception>(func, e => defaultValue);
}

Now I can write this:

result = Helper.Eval(() => a.b.c.d, null);

As an aside, it's really annoying not being able to write something like this:

public static T Something<T>(Func<T[]> func)

instead of writing say 5 overloads for Func<T> to Func<T1, T2, T3, T4, T>.

More on ServiceLocator - caching

2009-01-03T01:15:00.002+02:00

I got an idea while watching this webcast with Rob Conery and Jeremy Miller - Jeremy was talking about StructureMap and how you can set caching with it. It took me a few hours but I managed to duplicate that functionality. Of course, it doesn't work when you call Discover() - that one will still be the regular un-cached call (I think Jeremy calls it PerInstance).

This is an usage example:

Bind<DataContext>
.To(() => new DBDataContext())
.CacheBy(InstanceScope.Singleton);
Bind<IProductRepository>
.To(() => new ProductRepository())
.CacheBy(InstanceScope.HttpContext);

Here are the helper classes in all their glory:

using System;
using System.Collections.Generic;
using System.Threading;
using System.Web;
namespace Helper
{
// usage:
// Bind<ISomething>
// .To(() => new Something())
// .CacheBy(InstanceScope.ThreadLocal);
/// <summary>
/// Fluent interface simplifying the registration of implementations
/// </summary>
/// <typeparam name="T">Type to implement</typeparam>
/// <example>Bind<IRepository>.To(() => new Repository());</example>
public static class Bind<T> where T : class
{
/// <summary>
/// Maps T to Func<T>
/// </summary>
/// <param name="func">Function returning an instance of T</param>
public static BindHelper<T> To(Func<T> func)
{
ServiceLocator.Register(func);
return new BindHelper<T>(func);
}
}
public class BindHelper<T> where T : class
{
private readonly Func<T> func;
public BindHelper(Func<T> func)
{
this.func = func;
}
public void CacheBy(InstanceScope scope)
{
ServiceLocator.Unregister<T>();
ServiceLocator.Register(Memoize(scope));
}
private static readonly Dictionary<string, object> cache = new Dictionary<string, object>();
private Func<T> Memoize(InstanceScope scope)
{
Func<Type, string> GetTypeKey;
switch (scope)
{
case InstanceScope.NoCaching:
return func;
case InstanceScope.Singleton:
GetTypeKey = type => type.ToString();
break;
case InstanceScope.ThreadLocal:
GetTypeKey = type => type + "_" + Thread.CurrentThread.ManagedThreadId;
break;
case InstanceScope.HttpContext:
GetTypeKey = type => type + "_" + HttpContext.Current;
break;
case InstanceScope.Hybrid:
GetTypeKey = type => type + "_" + HttpContext.Current ?? Thread.CurrentThread.ManagedThreadId.ToString();
break;
default:
GetTypeKey = type => type.ToString();
break;
}
return () =>
{
object result;
if (!cache.TryGetValue(GetTypeKey(typeof(T)), out result))
cache[GetTypeKey(typeof(T))] = func();
return cache[GetTypeKey(typeof(T))] as T;
};
}
}
}

By all means, if you can figure out a way to simplify this, let me know :)

Ah, I forgot the InstanceScope enum:

namespace Helper
{
public enum InstanceScope
{
HttpContext,
Hybrid,
ThreadLocal,
Singleton,
NoCaching
}
}

I couldn't possibly say this any better...

2008-12-31T01:06:00.001+02:00

Comment by LKM on http://www.codinghorror.com/blog/archives/001201.html:

About convenience:

Pirated media is not free to the pirate. He pays with his time. Now you can't compete with the price of time for people who don't work. To most teenagers, time is way less valuable than money. No matter how convenient you make buying your stuff, these people won't give you their money. However, that is not the case for people who have jobs. To people like us, even finding a working torrent file may not be worth the hassle if we can instead open the Amazon MP3 store, type the name of the song we want and click "buy".

In many cases, the price doesn't even enter the equation. To me, whether I have ten bucks more or less on my credit card statement at the end of the month simply doesn't matter at all. What matters is whether I spend ten minutes looking for a song, or ten seconds.

Ironically, DRM makes it *less* convenient to buy, thus giving potential customers more incentive to pirate.

About instant gratification:

If possible, allow users to download whatever you're selling. Don't make them wait for the Amazon package to arrive if you can avoid it. Give them a file they can download right after clicking "buy", and don't make them pay additionally for the privilege.

Repository pattern

2008-12-30T19:50:00.007+02:00

[Edited on June 6, 2009.]

This is my implementation of the Repository pattern in C#; I am using the ServiceLocator defined in my previous post.

This is the main interface:

public interface IRepository<TEntity, TId>
where TEntity: IEntity<TId>
{
/// <summary>
/// Returns all entities in the repository
/// </summary>
/// <returns>List of entities</returns>
IEnumerable<TEntity> GetAll();
/// <summary>
/// Returns all entities matching a criteria
/// </summary>
/// <param name="criteria">Criteria to be matched</param>
/// <returns>All entities matching the criteria; should not be null, but it can be an empty list</returns>
IEnumerable<TEntity> FindAll(Expression<Func<TEntity, bool>> criteria);
/// <summary>
/// Returns the first entity matching a criteria
/// </summary>
/// <param name="criteria">Criteria to be matched</param>
/// <returns>The first entity that matches a criteria; can be null</returns>
TEntity Find(Expression<Func<TEntity, bool>> criteria);
/// <summary>
/// Finds a record with a given id
/// </summary>
/// <param name="id">Key to search</param>
/// <returns>The record with the given id or null if none exists</returns>
TEntity Find(TId id);
}

Now, I am always creating a primary key for each table, and nowadays I call it Id. It's normally an autoincrementing integer, but for users or other sensitive information I will probably switch to GUIDs, so the type of the primary key should be a parameter. Here is the entity interface, representing a record in a table:

namespace Helper
{
public interface IEntity<TId>
{
TId Id { get; set; }
}
}

As you can see, the only relevant thing is the primary key :)

I've extracted the updates to another repository... this is incompletely refined, you might need more granularity (some repositories might support updates but not deletes, for example).

public interface IUpdateableRepository<TEntity, TId>
where TEntity : IEntity<TId>
{
/// <summary>
/// Adds an entity to the repository; the repository may change the entity's Id on adding
/// </summary>
/// <param name="entity">Entity to add</param>
void Add(TEntity entity);
/// <summary>
/// Deletes an entity
/// </summary>
/// <param name="entity">Entity to delete</param>
void Delete(TEntity entity);
/// <summary>
/// Updates an entity
/// </summary>
/// <param name="entity">Entity to update</param>
void Update(TEntity entity);
/// <summary>
/// Commits all changes made so far
/// </summary>
void Commit();
}

Finally, this is a possible implementation of a repository, using the DI pattern (and the ServiceLocator class):

public class Repository<TEntity, TId> : IRepository<TEntity, TId>
where TEntity : class, IEntity<TId>, new()
{
protected readonly IEnumerable<TEntity> data;
public Repository(IEnumerable<TEntity> data)
{
this.data = data;
}
#region IRepository<TEntity>
public virtual IEnumerable<TEntity> GetAll()
{
return data;
}
public virtual IEnumerable<TEntity> FindAll(Expression<Func<TEntity, bool>> criteria)
{
var cond = criteria.Compile();
return from entity in data where cond(entity) select entity;
}
public virtual TEntity Find(Expression<Func<TEntity, bool>> criteria)
{
var cond = criteria.Compile();
return (from entity in data where cond(entity) select entity).FirstOrDefault();
}
public virtual TEntity Find(TId id)
{
return Find(entity => entity.Id.Equals(id));
}
#endregion
}

public class LinqRepository<TEntity, TId> : Repository<TEntity, TId>, IUpdateableRepository<TEntity, TId>
where TEntity : class, IEntity<TId>, new()
{
protected readonly DataContext db;
public LinqRepository()
: this(ServiceLocator.GetInstanceOf<DataContext>())
{
}
public LinqRepository(DataContext db)
: base(db.GetTable<TEntity>())
{
this.db = db;
}
#region IRepository<TEntity>
#endregion
#region IUpdateableRepository<TEntity>
public void Add(TEntity entity)
{
((Table<TEntity>) data).InsertOnSubmit(entity);
}
public void Delete(TEntity entity)
{
((Table<TEntity>) data).DeleteOnSubmit(entity);
}
public void Update(TEntity entity)
{
((Table<TEntity>) data).Attach(entity, true);
}
public void Commit()
{
db.SubmitChanges();
}
#endregion
}

Note: since I initially wrote this, I have read a few more articles discussing the need of using the Repository pattern at all. Here's a list of pro and con articles... decide for yourself :)

Ayende's article, first one I read on the subject.
Greg Young's first, second and third article.
Richard Dingwall's article.
This article suggests an interesting split - you should have two domains, one for changing your domain and one for querying it, and the two would have very different repositories. I have to think about it :)

ServiceLocator redivivus

2008-12-30T14:15:00.013+02:00

This is the (so far) final form of the ServiceLocator class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
namespace Helper
{
public static class ServiceLocator
{
private static readonly Dictionary<Type, Func<object>> container = new Dictionary<Type, Func<object>>();
private static readonly object containerLock = new object();
public static Func<Type, object> OnTypeNotFound =
type => { throw new NotSupportedException("Cannot resolve type " + type); };
/// <summary>
/// Discovers all the classes decorated with the Register attribute and registers them as implementations of the ImplementedType argument
/// </summary>
public static void Discover()
{
var types = Assembly.GetCallingAssembly().GetTypes().ToList();
types.ForEach(type =>
{
var attr = type.GetCustomAttribute<RegisterAttribute>();
if (attr != null)
Register(attr.ImplementedType, () => Activator.CreateInstance(type));
});
}
/// <summary>
/// Helper function to return a custom attribute of a specified type
/// </summary>
/// <typeparam name="T">Desired type for the custom attribute</typeparam>
/// <param name="type">Type searched for the custom attribute</param>
/// <returns>The custom attribute or null if it was not found</returns>
private static T GetCustomAttribute<T>(this ICustomAttributeProvider type) where T : class
{
return type.GetCustomAttributes(typeof (T), true).FirstOrDefault() as T;
}
/// <summary>
/// Maps the type T to a function returning T
/// </summary>
/// <typeparam name="T">Type requested by the application</typeparam>
/// <param name="func">Function returning an instance of T</param>
public static void Register<T>(Func<T> func) where T : class
{
Register(typeof (T), () => func());
}
/// <summary>
/// Maps the type T to a function returning T
/// </summary>
/// <param name="T">Type requested by the application</param>
/// <param name="func">Function returning an instance of T</param>
public static void Register(Type T, Func<object> func)
{
if (Contains(T))
throw new ArgumentException("Type " + T + " already registered.");
lock (containerLock)
container.Add(T, func);
}
/// <summary>
/// Removes the association for type T (for example as a preparation for another registration)
/// </summary>
/// <typeparam name="T">Type to unregister</typeparam>
public static void Unregister<T>() where T : class
{
Unregister(typeof (T));
}
/// <summary>
/// Removes the association for type T (for example as a preparation for another registration)
/// </summary>
/// <param name="T">Type to unregister</param>
public static void Unregister(Type T)
{
lock (containerLock)
container.Remove(T);
}
/// <summary>
/// Returns true if the type T is registered
/// </summary>
/// <typeparam name="T">Type to check for an existing registration</typeparam>
/// <returns>True if T is registered</returns>
public static bool Contains<T>() where T : class
{
return Contains(typeof (T));
}
/// <summary>
/// Returns true if the type T is registered
/// </summary>
/// <param name="T">Type to check for an existing registration</param>
/// <returns>True if T is registered</returns>
public static bool Contains(Type T)
{
lock (containerLock)
return container.ContainsKey(T);
}
/// <summary>
/// This is used by the application when it needs an instance of T
/// </summary>
/// <typeparam name="T">Type that the application needs an instance of</typeparam>
/// <returns>An instance of T</returns>
public static T GetInstanceOf<T>() where T : class
{
return GetInstanceOf(typeof (T)) as T;
}
/// <summary>
/// This is used by the application when it needs an instance of T
/// </summary>
/// <param name="T">Type that the application needs an instance of</param>
/// <returns>An instance of T</returns>
public static object GetInstanceOf(Type T)
{
lock (containerLock)
{
Func<object> result;
if (container.TryGetValue(T, out result))
return result();
}
return OnTypeNotFound != null ? OnTypeNotFound(T) : null;
}
}
}

Features:

OnTypeNotFound() - function mapping a type to an object if there is no explicit mapping for it. By default it will throw an exception, but you can replace it if you want, for example, to automatically Mock all types not explicitly mapped.

Discover() - this will inspect the calling assembly and automatically register as implementations all types decorated with the Register attribute.

Register() - maps a type (normally an interface type, but that's not required) to a Func<> returning an instance of that type. This allows, for example, a singleton implementation where the Func<> always returns the same object. Register() will throw if the type was already registered to avoid overwriting a registration by mistake; for intentional overwriting, use Unregister() first.

Unregister() - removes an existing mapping; does not throw even if one does not exist.

Contains() - can be used to verify that a type is registered, as GetInstanceOf() will by default throw if it's not.

GetInstanceOf() - this finds the function mapped to the given type and executes it, returning its result; if a mapping for this type is not found, GetInstanceOf() returns OnTypeNotFound() - which implicitly throws but can be overwritten.

The automatic registration (the Discover() method) needs implementation classes to be decorated with the Register attribute:

using System;
namespace Helper
{
/// <summary>
/// Use this attribute to decorate classes that should be registered as implementations of the argument
/// </summary>
/// <example>[Register(IRepository)]</example>
[AttributeUsage(AttributeTargets.Class, AllowMultiple = true)]
public class RegisterAttribute : Attribute
{
public Type ImplementedType { get; set; }
public RegisterAttribute()
{
}
public RegisterAttribute(Type implementedType)
{
ImplementedType = implementedType;
}
}
}

Finally, I have also created a helper class for manual registration:

using System;
namespace Helper
{
/// <summary>
/// Fluent interface simplifying the registration of implementations
/// </summary>
/// <typeparam name="T">Type to implement</typeparam>
/// <example>Bind<IRepository>.To(() => new Repository());</example>
public static class Bind<T> where T : class
{
/// <summary>
/// Maps T to Func<T>
/// </summary>
/// <param name="func">Function returning an instance of T</param>
public static void To(Func<T> func)
{
ServiceLocator.Register(func);
}
}
}

Decorating a class is obvious:

[Register(typeof (DataContext))]
public partial class DBDataContext
{
}

In the initial part of the application (like Global.asax.cs) you can do either this:

ServiceLocator.Discover();

or this:

Bind<DataContext>.To(() => new DBDataContext());
Bind<IProductRepository>.To(() => new ProductRepository());
Bind<IImageRepository>.To(() => new ImageRepository());

Finally (!), replace all "new class()" with:

var repository = ServiceLocator.GetInstanceOf<IProductRepository>();

In fact, it is recommended that you do not "hide" the dependencies, which means creating (usually) two constructors: one explicitly indicating dependencies and a default one calling the first, like this:

public class ProductController : Controller
{
private readonly IProductRepository repository;
public ProductController() : this(ServiceLocator.GetInstanceOf<IProductRepository>())
{
}
public ProductController(IProductRepository repository)
{
this.repository = repository;
}
}

Whew :) Please let me know if this helps, or if you find any problems with it :)

5-minute dependency injection container in C#

2008-12-28T20:43:00.025+02:00

I found Misko Hevery's talks on the usefulness of Dependency Injection to be quite persuasive, so I have started using constructor DI in my projects. Unfortunately, some of those involve ASP.NET WebForms applications, and even in the one I just started using ASP.NET MVC there are classes I do not create myself (like the controllers). This means that when I write something like

public class ProductController
{
private ProductRepository repository;
public ProductController()
{
repository = new ProductRepository();
}
}

I am hard-coding a dependency on the concrete ProductRepository class inside ProductController.

There are (at least) two solutions to this. One of them would be to create a property in the ProductController class:

public IProductRepository Repository { get; set; }

and change the constructor to

public ProductController()
{
Repository = new ProductRepository();
}

This is the usual property dependency injection, and it works quite well. However, I find it annoying to do this for every class, not to mention that I don't like the idea of adding a public property just so I can test the class, since this breaks the information hiding tenet.

A variant of this option is to have two constructors:

public ProductController(): this(new ProductRepository())
{
}
public ProductController(IProductRepository repository)
{
Repository = repository;
}

A second option would be to use a DI container and do this:

public ProductController()
{
repository = Container.Get<IProductRepository>();
}

and somehow instruct the program, in the initialization part (like Global.asax.cs), to return a new ProductRepository instance every time I ask for an IProductRepository.

This seems to me a bit more fluent, however the association part in (most of) the existing DI containers seem to be overkill.

So... I came up with a home-made DI container by using the pattern in my previous post:

public class GenericContainer
{
private static readonly Dictionary<Type, Func<object>> container =
new Dictionary<Type, Func<object>>();
public static void Register<T>(Func<T> func) where T : class
{
container.Add(typeof (T), () => func());
}
public static void Unregister<T>() where T : class
{
container.Remove(typeof (T));
}
public static T Get<T>() where T : class
{
return container[typeof (T)]() as T;
}
}

(The Register() method uses a trick here to convert a Func<T> to a Func<object> so that it can be saved in the dictionary - I should be able to get rid of this once we get covariance / contravariance in .NET 4.0 and Visual Studio 2010.)

The usage is quite simple - this call will associate a type with a function returning instances of that type:

GenericContainer.Register<IProductRepository>(() => new ProductRepository());

and this call will replace the "new ProductRepository()" call:

GenericContainer.Get<IProductRepository>();

(Of course, I don't have to use interfaces here; I can use any type.)

Note that the flexibility of the function allows me to also handle the use cases where I want a singleton returned:

var productRepository = new ProductRepository();
GenericContainer.Register<IProductRepository>(() => productRepository);

During testing I will want to replace the object being returned with a mock object I control:

GenericContainer.Register<IProductRepository>(() => new MockProductRepository());

It's that simple!

As you can see, I'm quite excited about this... it might not have all the bells and whistles of a mainstream DI container, but I find it quite good for what I need.

Edit: Ok, this might be more of a Service Locator pattern instead of Dependency Injection... I am unclear on the terminology. Useful in any case, though :)

Edit 2: It looks like this pattern does have a problem: it hides dependencies. The constructor dependency injection shows clearly that class A depends on classes B, C, and D. Using a service locator hides that. I think the compromise suggested in the article - create two constructors, one exposing the dependencies and one using the service locator - is the best way for now.

Important pattern in C#

2008-12-11T12:42:00.006+02:00

While programming, one sometimes needs static methods - methods that cannot meaningfully be associated with an object (a class instance). One example that comes to mind is Math.Abs, but Factory.CreateCustomer might be another. On the other hand, static methods make the methods that use them hard to test - what do you do if you try to test a method that calls Factory.CreateCustomer()? You don't want it to do that, you might want to return a fake customer, or null, or a NullCustomer - any of a number of choices.

One solution - the most recommended one, in fact, as far as I can tell - is to extract the method in an interface (IFactory.CreateCustomer) and pass an instance of that interface to either the constructor of the class under test, or directly to the tested method. Which is nice, except sometimes you can't alter the constructor of a class (like Pages in ASP.NET and Entities when working with Linq to Sql), and passing the interface to each method looks somewhat ugly.

A few days ago I found another method (I think it was on Ayende's blog, but I'm not sure):

public static class Factory
{
public static Func<int?, Customer> FindCustomer = customerID => GetDB().FindCustomer(CustomerID);
}

This declares FindCustomer as a delegate to a function taking a nullable int (the customer ID) and returning a Customer, and initializes this delegate to point to a specific function. However, and here is the great part, nothing prevents one from overwriting this function in testing to return a mock customer or whatever.

It's too early for me to tell if this is a great idea or a bad one, so I might change my mind later; but so far I'm pleased with it.

Biblical laws

2008-11-22T00:27:00.005+02:00

During the last few years I have slowly come to a conclusion: the reason for a lot of Biblical laws is not actually religious (as in, God has decided that something should not be done and that's that); it is the fact that non-religious commandments would have required explanations beyond the Jews' knowledge at the time or at least that they would have not obeyed them for long otherwise.

Take incest, since it does make for a rather glaring weirdness. Most people - an overwhelming number - consider incest to be "yuck"... the term cannot be discussed in polite company, only among porn freaks. When people ask "who did Adam and Eve's sons marry?" and receive the obvious answer "their sisters", they go "that's disgusting".

And yet... incest was not forbidden for a long time in the Bible. A quick search on Google reveals this page where we can see that Abraham married his half-sister, Moses' father married his aunt, and even when incest started to be prohibited it started with father-child relationships and only later were siblings added.

Why is this? Well, why are incestuous relationships prohibited today, even for non-religious people? As it happens sometimes, the state actually has some sort of justification here: people seldom pay attention to reproduction until it's too late and children between close relatives will have a much higher probability of genetic defects. So, since prohibiting only the "having children" part is difficult and would require genetic testing for each birth - and then, what do you do with an "illegal" baby? - the state found it much easier to prohibit the sexual relationship altogether.

Now. Genetic defects. What exactly would be the way to explain that to a tribe of Jews five thousand years ago? How can you tell them "if you do this, in the long run (generations) your tribe will die out"? Well, making it a religious commandment would be one way :)

If that is the case, why wait a few thousand years? Why not start prohibit this right away? Well, the reason is very likely the catastrophic diminishing of the gene pool at the Flood. Prior to that we have no way of knowing how good the genetic material was, but we can safely bet that the eight saved people on Noah's boat were not the most perfect specimens - and even if they were, it would still be too small a population. Which meant that genetic defects would very soon start to propagate... and what do you know, the average age starts falling rapidly after the Flood, from around a thousand years to around a hundred. (Incidentally, this means that living to a thousand years is a viable possibility for humans, we just have to discover what got lost.) Therefore, the most likely conclusion is that the prohibition was not needed before - there were few enough defects that the chance of two parents having the same recessive one (thus increasing the chance of it being dominant in a child) was too small to matter.

What about another example: basic hygiene. Read Leviticus 15 and get stunned by the detailed instructions. Would a plain "wash yourselves so you don't get sick" have worked? In fact, there's even a mention of the reason for this commandment: "You must keep the Israelites separate from things that make them unclean, so they will not die in their uncleanness". How could you persuade a bunch of stubborn tribesmen to obey it? Why, make it religious: "... for defiling my dwelling place, which is among them".

What about having a day of the week off? Well... in 1922 Ford reduced the workweek to 48 hours (see Wikipedia) so it was even longer before then. That's in the richest country in the world. Can you imagine any employer granting an employee a non-working day a thousand years ago, let alone five thousand, unless it was a religious commandment?

Ok... wrapping it up. It seems that whoever wrote (or at least directed the writing of) the Bible had a lot of knowledge for the time. Prophecies can be claimed to have been added after the fact; how do you add genetics and hygiene and worker rights in a thousand-years old book?

More on incentives

2008-11-20T14:20:00.008+02:00

While watching a concert I started thinking about the amazing cooperation needed for such a task - there are dozens of people on stage, playing various instruments in very specific ways, and hundreds of people in the audience, whose only responsibility for a successful concert is to stay quiet during performances and maybe applaud in between. The performers are rewarded financially if they perform well, and they risk being fired if they don't. The audience risks being fined or even jailed if they don't behave.

Given all that distinction - disassociation on one hand, jail on the other - who do you expect to cause more intentional trouble on the average? My money is on the audience, of course [grin].

Market incentives work a lot better than force incentives. To me, this is a beautiful example of capitalism versus socialism. Even though the risks of disobeying the state are much greater than the risks of disobeying an employer or a business associate (there's no meaningful distinction between employer and customer), cooperation is still more likely to be encountered in the market.

Socialism does not scale

2008-11-20T13:57:00.004+02:00

I was deleting some spam and reflecting on the whole issue... as I was telling someone a few weeks ago, the problem with spam will be stopped when it will become a serious issue. Right now, the problem is distributed: a few benefit and a lot have to pay for it. (Just like in politics [grin].) Worse, there is a socialism of sorts in the relationship between internet providers: the sender doesn't have to pay the carriers for the traffic it sends through them. This messes up the incentives completely.

A simple scheme like a hundredth of a cent per email would make it too expensive to send bulk emails ($100 per million emails) while not affecting regular users in any way; legitimate companies would have already a large enough internet subscription that any email charge would be drowned in the noise; finally, large, voluntary mailing lists like RISKS would need something on the order of maybe a few dollars a day - again, irrelevant when compared with the price for the actual equipment and internet service fee.

Furthermore, this scheme would create the proper incentives - the ISPs would penalize other ISPs who allow spam through them, the PC owners who don't secure them and let their PCs become spambots would suddenly have a financial interest to fix the problem, which would in turn put pressure on the OS makers (well, on Microsoft [grin]) to improve their software and so on.

Socialism works in small settings, like the family or small comunities, which is probably why it seems so natural to people. Free markets are much better at solving large-scale situations though. They can be thought of as real-life optimization machines - neural networks made of people, if you will.

Extracting named variables from an array

2008-11-08T09:29:00.005+02:00

My code uses the following pattern in a lot of places:

string[] array = GetSomeData();
Debug.Assert(array.Length() >= 3);
m_Member = array[0];
m_Password = array[1];
m_SomeOtherInfo = array[2];

... and so on. This is happening often enough that I tried finding a clearer way of expressing it. I even asked a question on stackoverflow. The answers I got aren't bad, but I came up last night with a solution I like more. (It doesn't mean it's the best, and it does look a little kludgy, but I like the way my code looks.)

using System.Diagnostics;
public static class ArrayExtractor
{
public static void Extract<T1>(this object[] array, out T1 value1)
where T1 : class
{
Debug.Assert(array.Length >= 1);
value1 = array[0] as T1;
}
public static void Extract<T1, T2>(this object[] array, out T1 value1, out T2 value2)
where T1 : class
where T2 : class
{
Debug.Assert(array.Length >= 2);
value1 = array[0] as T1;
value2 = array[1] as T2;
}
}

This only shows the code for 2 values, I'm going to write it for up to 10 values or so. It's not as elegant as the other solutions but now I can simply write

string fileName;
string contents;
ArrayExtractor.Extract(array, out fileName, out contents);

or even better

array.Extract(out fileName, out contents);

Comments are evil

2008-09-12T11:28:00.005+03:00

I just spent an hour hunting a "blablabla.ini" file all over the drive because that's what the comments said it was using. At some point, I gave up and started reading the code. It was actually taking the application name and replacing the ".exe" with ".ini". Found that file in two seconds.

Misquoting House: comments lie.

The verb "is"

2008-08-13T08:36:00.006+03:00

When arguing about something, the verb "is" should be avoided as much as possible because of its ambiguity. "A is B" can represent:

Identity: life is life.

Membership to a class: a car is an object.

An attribute: this apple is green.

A relationship: 5 is greater than 3.

A modification of a verb: John is walking.

... probably other meanings I'm forgetting now.

Of course, in colloquial language "is" cannot be avoided and is not normally a problem.

Symmetry

2008-07-30T10:44:00.004+03:00

Just wanted to save a nice expression I read about yesterday:

1 << n >> 1

The value returned is 2^n/2, or 2^(n-1). I just thought it looked lovely :)

On 0.999...

2008-07-17T07:25:00.006+03:00

While trying to explain to someone that 0.999... aka 0.(9) is equal to 1 - not almost equal, but equal - one of the "arguments" he used was that recurring decimals are not really numbers, you can't do mathematical operations with them. While the idea is mind-boggling - 0.(1) is 1/9 which is a rational number - I thought of a proof that doesn't use recurring decimals: using base 9.

0.(1)₁₀ = (0.1)₉

(0.0)₉ + (0.1)₉ = (0.1)₉
(0.1)₉ + (0.1)₉ = (0.2)₉
(0.2)₉ + (0.1)₉ = (0.3)₉
(0.3)₉ + (0.1)₉ = (0.4)₉
(0.4)₉ + (0.1)₉ = (0.5)₉
(0.5)₉ + (0.1)₉ = (0.6)₉
(0.6)₉ + (0.1)₉ = (0.7)₉
(0.7)₉ + (0.1)₉ = (0.8)₉

Finally,

(0.8)₉ + (0.1)₉ = ?

Well... in any numeration base, when you finished the digits you restart at 0 and add 1 to the next order, so:

(0.8)₉ + (0.1)₉ = (1.0)₉

Recasting the above in base 10, we have

0.(8) + 0.(1) = 1.0

However, in base 10 we still have a digit left, so it is equally valid to write

0.(8) + 0.(1) = 0.(9)

just as it is valid to write 0.(7) + 0.(1) = 0.(8).

Given that the left side is identical, it follows that 1 = 0.(9). QED.

You want horror? Here's horror!

2008-07-15T16:46:00.008+03:00

Jeff Atwood latest article, Maybe Normalizing Isn't Normal is a real beauty. If you like idiots, that is. (Cue Fabian Pascal.)

This is by far the worst Jeff has ever wrote. I cannot believe that someone who claims to offer advice to programmers can say anything but "NEVER denormalize". For fuck's sake, educate yourself before you discuss a subject as anything but "hey, I heard somewhere that shooting yourself in the foot is a good idea, what do you think?". (Maybe I should have realized how smart he is when he associated with Joel "exceptions are evil" Spolsky.)

Let me quote a comment that explains the problem very succinctly:

Speaking from long experience, if you don't normalize, you will have duplicates. If you don't have data constraints, you will have invalid data. If you don't have database relational integrity, you will have orphan "child" records, etc. Everybody says "we rely on the application to maintain that", and it never, never does.

A. Lloyd Flanagan on July 15, 2008 06:34 AM

Jeff, stay away from databases. If your programming skills are similar to your logical skills, stay away from programming too...

Oh... let me start checking the articles he uses to support his idea.

First article

Our feeling is that this is ultimately far more scalable than black-box clustering.

Wow. Imagine that. Their feeling. Definitely a good argument.

The third is already ridiculous: they denormalize the database for "performance" and then discover the reason normalization exists: they lose consistency and they have to write a lot of code (that, one would assume, will take a lot of time) to fix that:

a lot of the flickr code infrastructure deals with ensuring data is consistent and well balanced and finding and repairing it when it's not

Note that: not if, when.

Good job. Yeah, denormalization ftw!

I'll stop here. I believe I made my point.

Static virtual in C#

2008-07-10T23:50:00.007+03:00

Argh, this is annoying. I need a Handler type with a virtual string GetProduct() method so that derived types "know" what product they are for. However, I don't want to have to create an instance of the actual class every time I need to ask for its product... I want it to be a virtual method. Too bad - C# doesn't allow virtual static methods.

What I ended up with was this: each class derived from Handler has a read-only static string property Product, each of them returning the product that class can handle. In the method where I need that product (when populating a drop-down for example) I use reflection to get the value of the property:

if (handlerType.BaseType != typeof (Handler))
return;
var product = (string) handlerType.GetProperty("Product").GetValue(null, null);

I'm not really happy with this - I miss Delphi's "class" type which allowed a much more elegant solution. I'll edit this if I find something I like better.

Edit: Heh, I found a solution I like better: attributes. I define the following class:

[AttributeUsage(AttributeTargets.Class)]
internal class ProductInfo: Attribute
{
public string Product { get; private set; }
public ProductInfo(string product)
{
Product = product;
}
}

Now I can decorate the handlers with a

[ProductInfo("xxx")]

attribute and extract the product name with

var attributes = handlerType.GetCustomAttributes(typeof (ProductInfo), true);
if (attributes.Length == 0)
return;
var product = ((ProductInfo) attributes[0]).Product;

First time I create a custom attribute. Helpful and elegant, so it won't be the last.

More on probabilities

2008-07-06T00:18:00.005+03:00

This is in regards to the often-repeated saying "improbable events happen all the time" (even Dembski says that... which only serves to show that even people on "my" side can be idiots).

I just read something that was so apropos of this:

Heroic Law of Probabilities: One-in-a-million chances happen 9/10

(Look for it on this page, it makes more sense in context.)

That is just too great for words, I will keep quoting that "law" every time the subject comes up. I have to read that story, even though I'm not a fan of Marvel stuff. (Plus, the main character has a Romanian name. Yay :P)

Speculation

2008-07-05T09:45:00.005+03:00

I believe that the mind and the brain are distinct things, while the mind and the soul are the same. However, memory is pretty clearly located inside the brain... which means that we lose our memory when we die. That would imply that the purpose of our life here is only to educate our soul... to transform us into better people. It also means that we do not know who we were or what we did - which would be good, because otherwise a lot (most? all?) of us would have to somehow come to terms with the shame of what we did. Not really compatible with heavenly bliss.

No basis in anything, really, just some idle speculation.

It's Raganwald's fault this time...

2008-06-28T23:33:00.005+03:00

I just spent a day trying to write a program to solve Raganwald's puzzle (at the end of the article). Until now I thought I was a very intelligent guy. Tough :P

At least I added a few more extension methods to my tools class :)

Reliability

2008-06-27T01:49:00.009+03:00

As far as I know, I've always been a proponent of "worse is better" - I favor "cheap and bad" over "expensive and good". I don't wear brand clothing, I don't use brand appliances and I definitely hate brand computers.

The price/performance ratio is one reason.

Case A. One $100,000 computer with a guaranteed up-time of 99.9999% - that's less than an hour of downtime in a year. (I sincerely believe that any actual computer with such a guarantee would cost a lot more than that.)

Case B. Several computers with a guaranteed up-time of 90% - that's one day of downtime in ten, or more than a month of downtime in a year.

How many computers are needed to obtain the same reliability as the first system?

Well... the chance of one such system failing is 1 - 90% = 10%. The chance of two of them failing at the same time is 10% x 10% = 1% (therefore, the up-time of two cheap systems has increased from 90% to 99%). Adding a third computer will increase that to 99.9%... and each new system added to the cluster will add another "9" to that figure. Which means that we only need six cheap computers for the same 99.9999% reliability.

Ok, this doesn't take into account the infrastructure needed when you have a cluster as opposed to a single system. Let's say that we want to be absolutely sure and use ten cheap computers for reliability, and that the cost of the additional infrastructure is the same as that of the computers. Which means that each individual computer needs to be cheaper than $5,000 in order for solution B to be more profitable than A. (And let's not forget that the expensive system needs a lot of infrastructure too - to guarantee 99.9999% up-time you need controlled cooling, a back-up generator and so on.)

$5,000 for a system that fails one day out of ten? The worst junk I ever used was a lot better than that, and I never paid that much for a system. I think $3,000 was the highest.

The moral of the story: do not underestimate the power of many cheap systems. Google is the best example :)

FanFiction.net

2008-06-26T13:37:00.006+03:00

I used to hate the FanFiction.net site because it doesn't have a "save the whole story" button. Which is annoying, because I want to use my ebookwise reader to read the whole book while sitting in bed, not on a chair looking at the screen.

So... at some point I got pissed bad enough that I felt like writing an application to do it. Since I don't actually have any readers for this blog :P I won't bother with documentation or anything. I've included both the source files and the executable in the archive here. (Yes, I know, I should get a domain, it's not that expensive. I'll do that if I ever decide to actually keep blogging.)

Zeitgeist

2008-06-26T13:30:00.006+03:00

I watched Zeitgeist the movie yesterday. Nice movie, I recommended it to a few friends for the 2nd and 3rd parts :) (Nice = it fits in with my previous beliefs, of course :P)

As for the first one... well, the whole Horus/Jesus thing is a hoax. This thread is a request by someone for actual evidence... now, the thread is huge, reading it all will take more time than seeing the movie :D, but the thing is, even Acharya's attempt at one point to substantiate her claims fails flat on its face. The conclusion - even by some atheists commenting on the thread - is that the whole thing is Massey's invention, which Acharya parrots because it sells books. (Who knew Dan Brown will practically spawn a new industry, huh?)

So... still a nice movie, and I did learn a lot even from the first part. (Yes, I have long known that the Catholic Church took over a lot of pagan holidays to make Christianity easier to swallow.)

A few helper methods in C#

2008-06-25T10:45:00.010+03:00

Just a few methods I find myself using in a lot of projects...

public static int ConvertToInt(this string value, int def)
{
int result;
return Int32.TryParse(value, out result) ? result : def;
}
public static int ConvertToInt(this string value)
{
return ConvertToInt(value, -1);
}

Usage:

string s = "123";
Console.WriteLine(s.ConvertToInt());

public static void ForEach<T>(this IEnumerable<T> set, Action<T> action)
{
Debug.Assert(set != null, "set");
Debug.Assert(action != null, "action");
foreach (var item in set)
action(item);
}
public static void ForEach(this IEnumerable set, Action<object> action)
{
Debug.Assert(set != null, "set");
Debug.Assert(action != null, "action");
foreach (var item in set)
action(item);
}

I find it much more expressive to use this ForEach extension method instead of a foreach loop:

list.ForEach(item => { ... do something with item ... });

rather than

foreach(var item in list)
{ ... do something with item ... }

It's probably just a quirk of mine, but I prefer it.

public static string AsXML(this object obj)
{
if (obj == null)
return "<NULL />";
string str;
Encoding utf8 = new UTF8Encoding();
var objType = obj.GetType();
var xmlatt = (XmlTypeAttribute) Attribute.GetCustomAttribute(objType, typeof (XmlTypeAttribute));
using (var ms = new MemoryStream())
using (var x = new XmlTextWriter(ms, utf8) {Formatting = Formatting.Indented})
{
var xs = xmlatt == null ? new XmlSerializer(objType) : new XmlSerializer(objType, xmlatt.Namespace);
xs.Serialize(x, obj);
str = utf8.GetString(ms.GetBuffer(), 0, (int) ms.Length);
}
return str;
}
public static object AsObject(this string xml, Type type)
{
using (var sr = new StringReader(xml))
return new XmlSerializer(type).Deserialize(sr);
}
public static T As<T>(this string xml) where T : class
{
return xml.AsObject(typeof (T)) as T;
}

Good when you need to (de)serialize pesky objects. Usage:

MyClass x = new MyClass();
string s = x.AsXML();
x = s.As<MyClass>();

Magic

2008-02-18T09:47:00.003+02:00

Wizard
Warlock
Sorcerer
Mage

Several words describing people who can use magic... even though none of them ever existed.

Right...

Too many people

2007-11-15T13:53:00.001+02:00

Overcrowding is a common fear these days, just like global warming and other similar crap. I just thought I'd save here a quick calculation to put things in perspective.

Living area for a single person: 100 m^2 (this would make it 400 m^2 for a 4-people family)

Number of people you can fit in a km^2: 10,000 (because 1 km^2 = 1 million m^2)

Land area of the Earth: 148 million km^2 (see Wikipedia)

Say only a fifth of that area is available for people, that would be 30 million km^2. Multiplied by 10,000 this gives us 300 billion people.

300 billion people. With no many-stories buildings, no overcrowding a la Asimov's fiction (how on Earth could he estimate only 40 billion people for Trantor I cannot imagine) and no food problems ('cause we have four fifths of the land area for that - in fact, even if we remove 30% of the total area as desert it still leaves us with 70 million km^2 for animals and crops).

One other thing - what the overcrowding gang never mention is this: on average, people will necessarily produce more than they consume (because they need to produce for their own consumption, for the new population and to improve the living standard). More people means more resources, not less - well, unless governments interfere with the process, of course.

Finally - these calculations are useless for persuading anyone. If it's not population, it's food. (Malthus said that people grow exponentially while food grows arithmetically, so we're going to run out of food sooner or later. That he was an idiot is not necessarily a surprise; that there are millions of other idiots who repeat that without realizing that food is also biological, and therefore should also increase exponentially, is the really weird part.) If it's not those, it's water (I'd try calculating the amount of water in a single glacier but it would also be pointless) or energy or who knows what else.

So. I guess the conclusion is: we're all gonna die. Yay :P

Non-nullable reference types in C#

2007-11-08T14:06:00.003+02:00

C# 1.0 has non-nullable value types (an int variable must always have a valid int value, it cannot be null) and nullable reference types (a string variable can be null). C# 2.0 added nullable value types (an int? variable - note the "?" - can be null). There are unfortunately no non-nullable reference types (a string variable which can never be null). Patrick Smacchia has a summary of a possible notation and also of the issues that could appear.

However, until Microsoft decides to add non-nullable reference types, there's a way of doing the same thing with a somewhat more complicated notation.

using System;
namespace Extensions
{
public struct NotNull<T> where T : class
{
public readonly T value;
public NotNull(T arg)
{
if (arg == null)
throw new ArgumentNullException("arg");
value = arg;
}
// convert non-nullable to regular
public static implicit operator T(NotNull<T> arg)
{
return arg.value;
}
// convert regular to non-nullable
public static implicit operator NotNull<T>(T arg)
{
return new NotNull<T>(arg);
}
// explicit cast to non-nullable -- needed when T == object
public static NotNull<object> Cast(object arg)
{
return new NotNull<object>(arg);
}
public override string ToString()
{
return value.ToString();
}
public override bool Equals(object obj)
{
return value.Equals(obj);
}
public override int GetHashCode()
{
return value.GetHashCode();
}
public static bool operator ==(NotNull<T> left, NotNull<T> right)
{
return Equals(left, right);
}
public static bool operator !=(NotNull<T> left, NotNull<T> right)
{
return !Equals(left, right);
}
}
}

A method enforcing a not-null argument would look like this:

public string Test(NotNull<string> s)
{
string ss = s;
return ss.Substring(0); // no need to test for null
}

and would be called like this:

public void TestNotNull()
{
string s;
s = "something";
Test(s);
s = null;
Test(s);
}

Incidentally, this notation also serves as a form of specification - this parameter should not be null. I was thinking of using attributes but I think it would lead to a more verbose notation in case of several non-nullable parameters. I hope this helps someone :)

Structure of Object-Oriented systems

2007-11-03T00:51:00.008+02:00

Objects are state machines

Real-life entities have state. Your checking account has an amount of money; your car has a mileage, a gas level, a speed and so on. Objects in an application should also have a state and a set of operations that can manipulate that state. (You can change the color of your car or add money to your checking account; you cannot do the opposite.)

That being said, we normally want to hide the state and expose operations because it reduces coupling, which in turn reduces the "brittleness" of the whole system.

Tell, don't ask

Objects shouldn't have getters and setters; this is sometimes called the "tell, don't ask" principle. There are two cases where getters appear to be necessary:

Asking an object for some information and then deciding what how to change the state of the same object. This code could be a fragment of a ping-pong game:

if (ball.Y < 0 || ball.Y > MAX_Y)
ball.SpeedY = ball.SpeedY * -1;

A much better way to do this is:

ball.CheckYBounds();

thus encapsulating logic having to do with the ball inside the ball object.

What about when you ask an object for its status and then modify *another* object depending on that status?

if (ball.Y < 0 || ball.Y > MAX_Y)
{
ball.SpeedY = ball.SpeedY * -1;
sound.Bell();
}

Hmm. Changing this to having the ball object invoke sound.Bell(); doesn't seem that good now. It creates a dependency between the ball object and the sound object. Even worse, what about drawing the ball on the screen? Allen Holub suggests having a method ball.DrawOnScreen(); - but this seems way too weird. The ball object is a "model world" object. It should only "know" about model world stuff - moving around, having coordinates, a size and a speed, interacting with other object. Having it know about drawing stuff seems dangerous. What if I want to save its state to a database? Should it know about that too? This is definitely a violation of the Single Responsibility principle (aka SRP) - an object should only have one reason to change.

Communication between objects

One thing I read in the Turbo Pascal 6 manuals was this: every time you need object A to communicate with object B, think about adding a new object X in between them. At first, it sounds weird. However, if you need to add objects C, D and E to the mix, and all of them need to communicate with each other, then it pays for all of them to go through X: this way, you only need to have "send to X" and "receive from X" algorithms, instead of all the combinations. The clipboard is a good example of this: Word doesn't need to know how to communicate with Excel, Notepad, and a zillion other programs; it only needs to know how to put stuff into the clipboard and get stuff out of it.

So - how can this solve the above problem? How can the ball object inform the sound object that it needs to sound the bell, without creating a dependency between the two?

My solution would be a messaging system. We can have the ball object invoke the messaging object, which in turn invokes the sound object.

void Ball.CheckYBounds()
{
//...
if (ball.Y < 0 || ball.Y > MAX_Y)
{
ball.SpeedY = ball.SpeedY * -1;
Messaging.SendMessage(SOUND_BELL);
}
//...
}
void Sound.OnBell() // answers to SOUND_BELL
{
//...
}

Ok... the ball object is now free of the dependency on the sound object... it doesn't even have to know about the Sound class. However, it still has to know about the SOUND_BELL message. Why would a Ball know about making sounds? What if I want to reuse the Ball class in another application, where it's not supposed to make sounds when it gets reflected by the up and down edges? What if it's also supposed to flash when this happens - should I change the Ball class when I need to make a change to the way the ball is displayed? We're back to violating the single responsibility principle.

Inform, don't direct

Here is a great way of removing all such dependencies: don't have the object decide what other objects should do; just inform them of the state change and let the interested ones react to it (or not):

void Ball.CheckYBounds()
{
//...
if (ball.Y < 0 || ball.Y > MAX_Y)
{
ball.SpeedY = ball.SpeedY * -1;
Messaging.SendMessage(BALL_HIT_HORIZONTAL_EDGE);
}
//...
}

This has a few beneficial effects:

The message sent by the ball is defined in the Ball class itself, where it makes sense. It made sense for the SOUND_BELL message to be defined in the Sound class, but in that case we still would have had a dependency from Ball to Sound, and the Ball class did not care about sounds. Any object that wants to react to the BALL_HIT_HORIZONTAL_EDGE message shouldn't have a problem about having a dependency on the Ball class - after all, it obviously cares about the Ball class. If we don't want the Sound class itself to have this dependency on Ball, that's fine: we can have a BallSounds object with dependencies on both Ball and Sound:

void Ball.CheckYBounds()
{
//...
Messaging.SendMessage(Ball.BALL_HIT_HORIZONTAL_EDGE);
//...
}
void Sound.OnBell() // answers to Sound.BELL
{
//... ring a bell
}
void BallSounds.OnBallHitHorizontalEdge() // answers to Ball.BALL_HIT_HORIZONTAL_EDGE
{
Messaging.SendMessage(Sound.BELL);
}

Note that both the compile-time and the run-time dependencies are in the directions we want: the Ball and the Sounds objects do not know about each other, and they also do not know about the BallSounds object. All the objects know only about the messaging system.

A second advantage is that we've created an actual plug-and-play architecture, which has been a long-held dream of programmers, as far as I know. Let's say we decided to log one or more (or all) messages flowing through the system for debugging purposes. Simply create a new object, subscribe it to all the relevant messages (maybe add an all flag to the messaging system for this purpose) and have it log them when invoked. None of the other objects need to be modified in any way for this.

This is the architecture I'm envisioning:

The important point to keep in mind is this: an object should not send messages telling other objects what to do; it should inform them on what has been done and let them react to that.

to be continued...

It's all Bram Cohen's fault

2007-10-24T11:41:00.005+03:00

Charles Petzold on why he doesn't sell as many books as he used to:

(Undoubtedly another factor contributing to crummy book sales is BitTorrent, but that's so painful an issue I can barely bring myself to discuss it.)

This is so much bullshit. Every idiot who comes up with this excuse should be required to pay a zillion bucks for a lecture by Eric Flint. The simple fact is that if a well-paid US computer programmer finds it cheaper to spend a few hours looking for a book torrent on the 'net and downloading it, then you're doing something really wrong. Where is the link to the ebook variant, Charles? Put it on the freaking web for 10 bucks and you'll have removed any incentive to look for a free version. Anyone still doing that would never have bought the paper version of your book anyway. (I paid $25 for The Tomes of Delphi and another $50, if I remember correctly, for shipping.)

Another problem, of course - as Eric points out - is that most people never know about a particular book. He has a very good example: go in a bookstore that has your book and see how many people see the book in a day. Not browse it, not buy it - just see the book. For all those who do not, it might as well not exist. That is why you need color and bullets - so that people talk about it and advertise it to their friends. Jeff Atwood's article was free advertising - too bad your book couldn't measure up to the competition.

As mentioned above, Eric Flint is way better at this than I could ever be.

Oh well. Ranting. What else is a blog for if not that? :P

Copyright considered harmful

2007-08-18T06:52:00.002+03:00

Like that's any news coming from me :) However, I just found this page - Unofficial Honorverse Mod in which (at the bottom) a Chris Roensch explains how his company cannot allow fans to create a free Homeworld mod based on the Honorverse because it would dilute their "intellectual property" rights (whoever invented this phrase should be shot) for their upcoming movies and games based on the same universe.

Of course, it will came as no great surprise that no such games and movies have been produced since the end of 2005... and I'm willing to bet that movies, at least, never will. (Not to mention that I'm also quite certain that any games that will be produced officially will suck big time. They usually do.)

So, another thing that does NOT get created because of the so-called IP rights. Yay for progress.

TDD, or not TDD?

2007-07-04T00:58:00.002+03:00

I'm pretty convinced that TDD (or better yet, BDD) is the way to go. As usual when I get to this stage, I no longer care a lot to discuss the issue. However, I just hit on an interesting article... one that I find particularly weird.

Ravi's point appears to be that TDD is rather useless... and / or that Peter Norvig is a much better programmer than Ron Jeffries. The thing is, as I said here I managed to solve the whole thing in about 8 hours total. Worse, I only did that once I started with a consistent test-first approach, my previous 3 or 4 attempts ended in failure.

So. While I have a huge ego (not so huge not to accept Peter Norvig as a better programmer than me though), I have strong doubts that I'm a better programmer than Ron Jeffries. I think that Vlad's article hits on part of a problem: discovering an algorithm using TDD is at least difficult, if not impossible - and TDD was not meant for that. I also think that Ron simply gave up - in fact, I also did that, several times - because the effort required to figure out the solution was not warranted by the importance of the problem. He started on what was probably the wrong idea and hasn't tried to get out of that box and re-think. If you will, he is guilty of having a (wrong) design up-front [grin].

(In my case, the issue was never the algorithm: I had in mind from my first attempt the final algorithm: try to discover "forced" values, where a cell can only have a single value, and repeat that until there's no such constrained cell left; after that, go with a "greedy" recursive algorithm where the first possible value of a cell is chosen, the whole algorithm is re-invoked, and if no solution is found go back and try the next possible value. I must admit that Peter's optimization "try the cell with fewest possible values first" never crossed my mind.)

So. In my opinion, TDD / BDD is quite useful... even for writing a Sudoku program :)

Databases and objects

2007-06-29T19:21:00.002+03:00

I have long been a fan of Fabian Pascal's database ramblings (not his political and economical ones, the man has absolutely no clue on those). Yes, the relational theory is actually set algebra, and therefore a relational database is the most complete and powerful type of database, bar none. Object databases, hierarchical databases (registry), network databases (objects) - they all have problems which simply do not exist in an RDB.

However, I was re-reading an article about the object / relational mismatch problem - Interoperability happens and I got to this part:

Developers simply give up on relational storage entirely, and use a storage model that fits the way their languages of choice look at the world. Object-storage systems, such as the db4o project, solve the problem neatly by storing objects directly to disk, eliminating many (but not all) of the aforementioned issues; there is no "second schema", for example, because the only schema used is that of the object definitions themselves. While many DBAs will faint dead away at the thought, in an increasingly service-oriented world, which eschews the idea of direct data access but instead requires all access go through the service gateway thus encapsulating the storage mechanism away from prying eyes, it becomes entirely feasible to imagine developers storing data in a form that's much easier for them to use, rather than DBAs.

So... an application could simply forget about the whole database thing and completely take over the storage and management of objects. It can also handle the problem (mentioned in that article) of other parts of the same enterprise wanting to use the data by exposing it through services, so that nobody can have access to the underlying objects (and thus nobody needs to change their own application if this application is refactored). True, such an "object database" lacks a lot of the powerful advantages of a relational one, like ad-hoc querying - but maybe for this application, You Ain't Gonna Need It.

Interesting idea. I think I read somewhere that Robert Martin starts what are to everyone else obvious database applications by ignoring that part... because he might not need a database after all, and he definitely does not need it to start with. I never understood that until now :)

Sudoku and TDD

2007-06-13T12:24:00.001+03:00

For the last few months, I've been trying on and off to write a program to solve a Sudoku game. The "easy" ones are easy, you just use a rudimentary algorithm:

each cell has a list of possible values associated with it

start by associating all values 1-9 to each cell

repeat until solved:

for each cell, remove any values already present in the same line, column, or box

By this process of repeated elimination you can solve a lot of the "easy" tables. However, this algorithm is too limited and it will go into an infinite loop if you have a situation where, for example, two cells in the same row can accept values 1 and 5, but at this point you have no way of knowing which is which.

Now, of course the solution to that problem is simply a recursive backtracking algorithm:

save the current state

pick one value and try to solve the rest of the table; if it worked, all is good; if not, go back to the saved state and try the next value

The devil, however, is in the details. I would go into some weird state where suddenly I had a row with two cells having the same value... which of course ruined everything else. And debugging something which is five recursion levels deep and goes through 81 cells each time... I wasn't willing to spend all that effort for a stupid problem, especially with the number of programs already available solving it.

That was the case until one night when I was really bored and wanted to program something for fun :) But I had also just read yet another article on Test-Driven Design and decided to try it. It took me a few hours that night, and a few more hours the next morning, but I managed to make it work. Hehe. That's a cool feeling, especially since, while I have tried TDD before, this was the first time I actually felt it helped.

That's code I'd take to an interview :)

Shape of the Earth

2007-05-14T22:19:00.001+03:00

One of the very stupid things you learn in school is about the shape of the Earth. You are told that the Earth is not exactly spherical - it has a "geoid" shape (which basically means it has a shape like the Earth... very intelligent). Like in the Microsoft joke, this is technically true, but rather pointless. The Earth's radius is 6500km; the deepest trench in the ocean is 11km and the highest mountain is almost 9km. Which makes it a pretty damn good sphere, with a precision of about 1:600.

Think about it this way: the same precision on a billiard ball with a radius of 3cm would mean a maximum error of 0.05mm, or 50 microns. I'd call that a perfect sphere...

Ricardo's Law

2007-05-09T01:03:00.002+03:00

While reading Snakes on a Plane (I'm a big fan of Buffy, and of Buffy crossovers) I got to this part:

Did they even have stores selling clothes and groceries and stuff back then, or did everybody make their own clothes and grow their own food? That seemed like a lot of wasted effort to Buffy. If you were good at making clothes or something, shouldn't that be, like, your job? And then you could sell the clothes you made or trade them to somebody who was good at growing things.

This is something that most people seem to understand intuitively. If you're better at making bread, and I'm better at making clothes, we should specialize and trade with each other, instead of each of us trying to make both. We'll have products of a better quality (or a greater quantity, depending on the meaning of "better" in the previous statement). This is why, for example, it makes sense to grow corn in Kansas and bananas in Kenya and trade, instead of each trying to do both.

What some people don't realize, because it is counter-intuitive, is that you don't even need an absolute advantage for trade to be beneficial. Consider the case of a good lawyer (ok, efficient lawyer [grin]) with a bad assistant:

the lawyer can get $200 an hour for legal advice

he can also type and file well enough to generate an income of $50 an hour

his assistant would only be able to get $10 an hour for legal advice

not being as good an assistant, he would also only do work worth $20 an hour

Even in this case, with the lawyer having a devastating advantage in both legal and secretarial work, it still pays off for him to only handle the legal advice and to hire the assistant for the rest. That's because of an important, but overlooked thing: during the time he's doing a good job typing or filing or whatever, he's not doing a great job getting paid for legal advice. In other words, for every hour spent doing the work of an assistant, he loses $150. Given that fact, it is better if he hires the assistant for $20 - he saves $130 for each hour he would have otherwise spent doing a less financially rewarding job.

More information can be found in the Wikipedia article... while I'm not a big fan of Wikipedia (way too many socialists in the world), this is actually a well-written article.

Predictions of creationism

2007-05-08T09:44:00.001+03:00

What would be true if creationism were true, but false if not? I don't mean "well, God would exist"; I mean what would we expect to happen differently?

Well, there's the whole "the universe is comprehensible" thing, which has been discussed in other places (and I do believe it's a great argument). But I just had an idea: the universe is huge. It has either been created by impersonal natural laws, or by God. In the second case, we expect it to have been created for our use, just like the Earth, and therefore at some point we will develop / discover a practical way of faster-than-light travel - just as we developed a way of crossing the oceans. Such an expectation is not warranted in the first case, since it's irrational to expect impersonal laws to care for our wishes.

So there. FTL technology. I can barely wait. grin

Ideas are worthless

2007-05-07T10:08:00.001+03:00

I have a friend who is normally a very intelligent man, and an extraordinary computer programmer. However, he has an IDEA, and he's unwilling to publish it in detail on his site - he wants to discuss it with a big company, to get the cash he believes he deserves for his idea.

I was unable to persuade him that ideas are worthless.

Here's an idea that would be worth a lot of money... if ideas were worth anything. Let's get rid of 99% of the existing government bureaucracy. (I'm an anarchist, I want to get rid of all government, but that's harder to sell. Most people would agree that 99% of the government is useless, though. grin)

Just think of the advantages:
* you save on salaries and bribes
* you force the leeches to actually work for a living and make something even marginally useful (like hamburgers)
* a lot less spending on "feel-good" projects like houses for the poor which cost a million bucks a piece and look pretty much like a shack
* ... and I'm sure a lot more, you get the point

What about getting rid only of the armies? I mean, most people are scared of guns. And if guns are bad, and people shouldn't have them, then the place we should start removing them is definitely the army - after all, we know that armies kill people, that's their job description :D

Now, are these ideas worth anything? Not even the time I spent writing them down. The implementation, on the other hand, would be worth billions. Get that perfected, sell it to the Americans, and become the richest man in the world. Eat your heart out, Bill Gates! grin

See? It's worthless to think "hey, I don't really like the links Yahoo returns, what about making a better search engine?". You have to actually work your butt off and do it to become Google. If you're the kind of person that has great ideas, you probably have a dozen a day. They're worthless to you, and they're worthless to everyone else - make a couple of them real, and everything changes.

Just a rant, of course.

Julian Simon

2004-04-15T14:17:00.001+03:00

In Ultimate Resource II, Julian Simon argues against the doomsayers that claim that everything will go from bad to worse. The ultimate expression of this gloomy vision, he says, is the second law of thermodynamics - which he argues against, by saying (approx. quote): "Where have they seen this? If we look around us, we will discover that things have, on the contrary, evolved from the smallest micro-organism to life as it is now. What second law? What we know of the world should show the opposite of entropy."

This is a classical example of reaching the right conclusion for the wrong reasons. There's no such thing as evolution. However, there is one thing that, each day, overcomes the second law: intelligence. Intelligence is the only known way of defeating entropy - of getting something more complex than what we put in.

Analogy

2003-09-15T15:00:00.001+03:00

A repeated accusation on the true origin list is that I incorrectly consider everything with a probability smaller than 10^-150 (but greater than zero) to be impossible. Well, I do, but is it incorrect?

The analogy I want to make here is between "any event with a probability smaller than 10^-150 is impossible (in this universe)" and "any cube with a length smaller than 10^-40 cm is impossible (in this universe)". Why the second claim? Because the Planck limit, 10^-33 cm, dictates how small objects can be - nothing can be smaller than this, in our universe. The universe is discrete at this scale - you can find to points between which there is no other.

So, saying "a cube with a length smaller than 10^-40 cm (or anything smaller than 10^-33 cm) is impossible" is not wrong; it is actually correct. By analogy, saying that "anything with a probability less than 10^-150 is impossible" is not wrong - it really is impossible in our universe.

Human DNA

2003-08-20T21:40:00.003+03:00

Is the probability of obtaining a DNA sequence which codes for a human (ie, a living being capable of interbreeding with humans) by any combination of random processes and deterministic functions (like natural selection) less than 10^-150?

Let's assume we already have the humans' alleged ancestor race, call it apes. Is it possible for random mutations to change an ape DNA into a human DNA?

Let S = {A, C, G, T} the possible values for a codon, and S* a sequence s[1] ... s[n] where s[i] is in S. We define

dist* (a, b) with a, b in S* = the number of point mutations needed to change a into b (or viceversa):

dist* (a[1] ... a[n], b[1] ... b[m]) = sum (i = 1, n, dist (a[i], b[i])) + m - n, with m >= n >= 1

dist (a, b) = {0 if a = b, 1 otherwise}, with a, b in S

In other words, if everything works out perfectly, it takes at least dist* (A, B) point mutations to convert one into the other, where A is a member of the ape DNA set, and B is a member of the human DNA set.

How close are these sets? It is alleged that they are very close - that the resemblance is between 95% and 99%. Out of 3 billion codons, this would give a difference of 30 to 150 million codons. If we go further and claim that only 1 in 10,000 of the DNA is actually relevant - the rest being "junk DNA" - we are left with a relevant difference of only 3 to 15 thousand "letters".

This means that we need between 3 and 15 thousand point mutations to get from apes to humans. How probable is that, using purely random mechanisms? Well, the problem is equivalent to getting from AAA...A to CCC...C, both strings having 3,000 letters, changing one letter at each step into one of the values A, C, G, or T - at random. At each step, the probability of getting the right one is 1/4; since we need 3,000 lucky changes, the probability of getting them through purely random means is 1/4^3,000 = 2^-6,000. This is WAY below 2^-500.

What about the Weasel program? (You can find an example here.) Couldn't such a program get much faster from AAA...A to CCC...C? Yes, of course it could. But the problem with the weasel program - or any similar algorithm - is that it already knows its target. In other words, the weasel program is equivalent to the following one:


print "CCC...C"

Unless someone can convincingly argue that evolution's goal was to transform apes into humans, any algorithm with a predefined target (which can be reduced to the above) is out. This includes more "roundabout" algorithms that try to conceal the fact that they know their target, like:


for i = 1 to 3000
  print chr(-i + ord("A") + (2 * i + 6) / 2 - 1)
next

What about using an evaluation function (for the natural selection part)? Those can serve as probability amplifiers (they lose information, instead of adding new one, but that's ok in this case - we already have a source of information, randomness; we now need something to "guide" that information).

Well, we have two relevant classes of such functions:
1) functions that know the target - and we're back to the above case - and
2) functions that calculate a "fitness", and it so happens that the fitness of the CCC...C string is greatest, while the fitness of the AAA...A string is lowest.

Option 2 is a viable one, provided that someone produces such a function for the real-life case of human DNA. (I know it's easy to do it for our strings.) Can this be done, without using any post-facto rationalization (like "humans are smarter than apes, therefore the function must give them more points")? [Remember, this must work for any pair of ancestor - evolved species. Such a function won't help with bacteria. The No Free Lunch theorems would suggest that there is no such function that works for all cases.]

So, the final conclusion would be: if evolution's goal is not to produce humans, and if there's no general algorithm that nevertheless happens to favor "closer-to-human DNA" over "closer-to-ape DNA", human DNA can't have occurred by chance - even with the help of natural selection.

Clarification

2003-08-14T22:08:00.002+03:00

For me, the idea of a string having a probability is absurd - I can't parse it at all. Let me clarify this with an example: what is the probability of a chair? Both a chair and a string are objects. Ok, the string is an informational entity, not a physical object. What is the probability of an equation? Neither of these questions make any sense.

Now, if we want it to make sense, we must start to expand the question. What is the probability that a 500-bit string will occur? Still not good enough - out of thin air? In my daily emails? So let's try again: what is the probability of a 500-bit string occuring in the following experiment: "write down 500 bits"? Well, 1 if you do it, 0 if you don't. In NO case is it going to be any other value.

How do you fill the blanks so that "what is the probability of a 500-bit string ..." gives you any other result? Please email me if you found a way.

Information

2003-08-07T15:46:00.002+03:00

Let's define I(E) = -log2 P(E), where E is an event. This is the amount of information contained in (imparted by) that event, or in other words the amount of uncertainty removed by that event, and is measured in bits. (Why uncertainty? Let {E*} be the set of possible relevant events, of which E is a member. Before E, any of the {E*} elements could have occured; the fact that we obtained E decreased that uncertainty. P(E) is, of course, 1 / the number of elements in {E*}.)

Using the value we determined earlier, the "cutoff value" of 10^-150, the information contained in an event E with that probability is -log2 10^-150, which is 500 bits. Therefore, another way of specifying the "point of no return" is this: anything with an informational content larger than 500 bits could not have occured without the intervention of an intelligence.

What are possible sources for this information? Well, as far as I know (please let me know if you find another one), only 3 exist: intelligence, randomness, and laws (deterministic functions). Of these 3, the last one doesn't actually create information (more on this below), but can be used to move it from one place to another. (Eg, a program drawing a rembrandt painting doesn't create the information: it copies it from whatever storage source it is using to the screen or printer.)

Why can't deterministic functions create information? Well, let's see:

P(A & B) = P(A) x P(B) -- where A and B are independent
I(A & B) = I(A) + I(B) -- this follows from I(A) = -log2 P(A)

However, we're interested in information produced by functions. That is, what is I(A & f(A)). Well, f(A) is definitely not independent from A, so we will use I(B | A) for "the (conditional) information of B, given A". From this, we obtain that

I(A & B) = I(A) + I(B | A) -- note that, for independent A and B, I(B | A) = I(B), so we get back to the previous formula

I(A & f(A)) = I(A) + I(f(A) | A)

What is I(f(A) | A)? It will be helpful to get back to probabilities:

I(f(A) | A) = -log2 P(f(A) | A)

But f is a deterministic function - that is, f(A) is completely determined by A; given A, f(A) has a probability of 1:

P(f(A) | A) = 1

which means that

I(f(A) | A) = 0

Getting back to our initial problem,

I(A & f(A)) = I(A) + I(f(A) | A) = I(A)

In other words, we obtained that deterministic functions add no information. (This is common knowledge in the field of cryptography, another hobby of mine, where it's expressed as "functions cannot create entropy".)

We have now an interesting result. Functions cannot add information. Randomness can, but no more than 500 bits. So the only conclusion we can draw is this: any piece of information more than 500 bits long (or, any event with a probability lower than 10^-150) must have an intelligent cause.

It is important that we keep thinking about the issue in terms of both information and probability, or we might get confused. For example, someone objected that it is possible to accumulate any amount of information "piece-by-piece", one bit at a time. None of the individual bits is outside the realm of randomness, but together they can break the 500 bits barrier. Or, it could be objected that a random process - like flipping a coin - can easily generate 500 random bits. We need to get back to the first part of this (Statistics) and talk about probabilities again. (The answer to both objections seems to me obvious; however, if you're of a different opinion, email me and I'll detail what I mean.)

2003-08-07T10:01:00.001+03:00

Statistics

I met this gem on the TrueOrigin list:

I asked her to pick a digit (0 through 9). Then do it again, and again, 53 times. You will have picked a number with 53 digits. The apriori probability that you would have picked that number is 1 chance in 10^53, but you did the impossible.

Is it true that the chance of doing what is described above is indeed very small (10^-53)? Let's see.

In my country, we have a lottery called "6 out of 49". You have a 7x7 grid, with numbers from 1 to 49, out of which you are supposed to pick the winning six. Let's say you bought a ticket and picked 6 numbers at random. What is the chance of winning the lottery?

Well, the total number of combinations is C(49, 6), which is 49! / (6! x (49 - 6)!), where "n!" (read: n factorial) means "1 x 2 x 3 x ... x n". Calculating it gives us 13,983,816 possibilities (if I made a mistake, please let me know). So, the probability of picking the right combination is approximately 1 / 14 million, or 7.15 x 10^-8.

Now, what is the probability that you picked 6 random numbers? Put this way, the question sounds silly: the probability is 1, of course. You just did it, it's a past event - it either happened or it didn't, so the probability is either 1 or 0. And this is what the above "experiment" is all about: picking 53 random numbers, and then asking "how probable is that you in fact picked 53 random numbers?", can only have one answer: the probability is 1.

How do we distinguish between the two cases? We need what is called a specification. The second case had one such specification: the winning combination. In other words, we must be able to ask the question: what is the probability that the sequence we just picked (call it x) is equal to T? We can do that in the second case; but in the first, we have nothing to replace T with - except for x, in which case the question becomes "what is the probability that x = x?". Once we have a specification, we can calculate the probability. Once we have that, we can determine whether the event was random or not: anything below a specific value (we'll use 10^-150) cannot be random. (More on this value later.)

What conditions must the specification meet? Well, only one: it must be independent. To borrow an example from Dembski, an archer that shoots at a wall, then paints a bullseye around the arrow is not a very good archer. (The probability of hitting the target - the probability of x being equal to T, to use the symbols above - is 1, since T is defined by x. The two are anything but independent.) On the other hand, if the bullseye is already painted, and the archer hits it 20 times right in the center, then we can be quite sure this was no random event. (I am, of course, ignoring various way to "rig" this, like a bullseye 20 meters across, with the archer standing two feet before it, and other variations.)

Should the specification occur before the event? No, this is not necessary. A process generating 3141592 will hint that it's not random. A process generating the first 100 digits of pi will absolutely tip us off. An independent specification is enough to calculate probabilities, whether it's made known before or after the fact.

Can we make mistakes? In other words, is it possible to consider that an event is random when it isn't, or the other way around?

1. We might not realize that there is a specification. A process generating 2030481... might seem random, until we realize that each digit is the corresponding digit of pi, minus one. Or, some message might be encrypted with a good algorithm, and the resulting string would appear to be random to all statistics tests (like diehard). False negatives are a possibility.

2. On the other hand, I don't see any way for false positives. If the "cutoff value" is low enough - like the 10^-150 I'm using (which I borrowed from Dembski) - there is no way it can happen randomly (that is, without an intelligence involved). [I am not a subscriber to the QM "anything possible is necessary" many-worlds interpretation.] Of course, with a cutoff value too large - like 1 in 10 - false positives can happen. This is why Dembski picked 10^-150, to avoid any possibility of a false positive.

Finally, why 10^-150?

It is estimated that there are about 10^80 particles in the known universe.
The Planck time is the smallest meaningful unit of time: 10^-45 seconds. This means that nothing physical can change state more than 10^45 times per second.
Finally, the universe is estimated to be more than a billion times younger than 10^25 seconds.

Multiplying these figures, 10^80 x 10^45 x 10^25 gives 10^150. This means that, if all the particles in the known universe, for a billion times longer than they (allegedly [1]) existed, "tried" to generate new combinations, they wouldn't have exhausted 10^150 such combinations. That is, there are simply not enough resources in this universe to account for an event with a probability smaller than 10^-150. Which means that, if we encounter an event whose probability (reminder: we need an independent specification to be able to calculate this probability) is smaller than 10^-150, an intelligence must have been involved.

In other words, if P(E) < 10^-150, E was caused by an intelligence.

Whew!

[1] I'm an young-earth creationist. I believe the universe to be less than 10,000 years old.