DateTimes Through OLEDB in VFP

For some time, we've been running queries against a SQL Server database through an OLE DB connection. Recently, I came across some strange behaviour when retrieving datetime values.

In one location, the application populating the SQL database allows for an empty date value. However, SQL Server doesn't allow for an empty datetime value (though it does support a null datetime). The application handles this by using the maximum SQL Server datetime value of 9999-12-31 23:59:59.997 to represent an empty date. The data looks something like this:

This is maybe a bit unusual, but not too strange. However, when querying this data through an OLE DB connection in VFP, this is the result:

That's curious. Why is the date appearing as a blank? Doing some quick investigating on the retrieved record, both Empty(ONDATE) and IsNull(ONDATE) return false. Even more curious. What would make a datetime value display as blank, but still evaluate as not empty and not null?

Ok, let's try connecting to the database using an ODBC connection instead. Here are the results of the same query:

Curious again. The data is appearing correctly here. Something must be different in the way that OLE DB and ODBC are handling these datetime values. Let's add a couple more testing values, and then retrieve them both ways, to see if that sheds any light on the situation.

This shows what's going on. The OLE DB connection is using the milliseconds to round the value to the nearest second, while the ODBC connection is just truncating the milliseconds. The weirdness with the blank value is coming from the fact that 9999-12-31 23:59:59 is also VFP's maximum datetime value, and the rounding is forcing the value past this maximum.

I generated a VFP table using the results of the OLE DB query, and opened it with a hex editor. Sure enough, there is data in the "blank" datetime value. VFP stores a datetime value in two pieces: the date as a Julian day, and the number of milliseconds past midnight. The problem value has a number of milliseconds that evaluates to slightly more than 24 hours. The Empty and IsNull functions are correctly reporting false, based on the fact that there really is a value stored there.

All in all, this is not too much trouble to work around, since we can just use an expression with CASE or DATEADD/DATEPART to have SQL Server adjust the value for us before sending the query results. It's good to understand this behaviour though, since it will also appear in other situations where milliseconds are included in datetime values, and it will be much less noticeable that any rounding is happening.

Pumpkin Carving Madness

For the past few years, we've held a pumpkin carving contest for Halloween. People come over and carve their pumpkins, and then the trick-or-treating kids & parents vote for their favourite. I won the first contest in 2006, but I haven't been able to repeat. Kathleen has won the past two contests.

Here are a few pictures from this year.

Pumpkin Carving Madness 2009 underway

Owen inspecting one of the pumpkins

My entry for this year, good for second place in the voting

Kathleen's winning entry

Owen at 6 Months

Here are some new photos of Owen. He's six months old now!

SQL Subqueries with Null Values

Null values in data can often cause unexpected results. Recently I came across a case where the field in a NOT IN subquery contained null values, and I didn't get the behaviour I was expecting. After spending some time with this, I have it worked out, and the behaviour does make sense. It's worth having a look at.

We commonly use a NOT IN subquery to retrieve records that do not have a related record in another table. The common example of this type of query is to retrieve all customers that do not have any orders. Here's a simple example of a Customers table and an Orders table:

Customers                Orders
CustID    FullName       OrderID   CustID    OrderDate
     1    John Doe             1        2    1/1/2009
     2    Jane Doe             2        3    1/2/2009
     3    Jack Smith           3        3    1/3/2009
     4    Jane Smith

There are four customers, and two of these customers have placed orders. We can use IN and NOT IN subqueries like these:

select * from Customers where CustID in (select CustID from Orders)
2 Jane Doe
3 Jack Smith

select * from Customers where CustID not in (select CustID from Orders)
1 John Doe
4 Jane Smith

This is as we would expect. However, let's add a fourth record to the Orders table, with a null value for the CustID:

OrderID   CustID    OrderDate
      4     null    1/4/2009

Now, when we run the IN query, the results are unchanged; we still get customers 2 and 3. However, when we run the NOT IN query:

select * from Customers where CustID not in (select CustID from Orders)
No records returned

Why are there no records returned? Shouldn't we still be getting customers 1 and 4, since these CustIDs do not appear in the Orders table? Well, let's look at how this gets handled. The subquery generates a list of CustIDs, like this:

select * from Customers where CustID not in ( 2, 3, null )

Logically, the NOT IN is treated as a series of not equals expressions, like this:

select * from Customers where ( CustID <> 2 and CustID <> 3 and CustID <> null )

Now, we can consider how this evaluates for our customer records. For customer 1, the where clause becomes:

1 <> 2 and 1 <> 3 and 1 <> null
= true and true and null
= null

This is why record 1 doesn't appear in the result set; for records to appear, the where clause must evaluate to true, not to null.

Now that I've gone through the logic on this, it's not really correct to say that customers 1 and 4 don't have any orders. The null CustID value in the Orders table means that the customer for that order is unknown, so we can't guarantee that this order doesn't belong to customer 1 or 4.

Stonefield Article on SoftwareCEO

Stonefield Software is the subject of an extensive article on SoftwareCEO.

Stonefield Profile in the Financial Post

The Financial Post published an article profiling Stonefield Software yesterday.

Prairie firm finds its markets

Stonefield Profile

An article profiling Stonefield Software, the company I work for, was on the front page of the business section in the Leader Post. The article is available on the Leader Post website.

Stonefield Software the little Regina company that could

Exposition Problems

Recently, there was a blog post on my feed reader, referencing a recent paper on the topic of Open Exposition Problems in mathematics. To introduce this term, I'll use the same quote from the paper as was given in that blog post:

All mathematicians are familiar with the concept of an open research problem. I propose the less familiar concept of an open exposition problem. Solving an open exposition problem means explaining a mathematical subject in a way that renders it totally perspicuous. Every step should be motivated and clear; ideally, students should feel that they could have arrived at the results themselves.

This is an interesting idea, and I think it has applications in software development as well. The normal approach when explaining an algorithm is to just explain its steps. For any reasonably complex algorithm, it's also required to give some justification for why these steps achieve the desired result. Generally, the idea is that the student obtains enough of an understanding of the logic to produce a working version of the algorithm, and to extend it if need be.

That's fine, but the quoted text above goes further. It talks not only about the problem itself, but also about the motivation behind the steps of the solution, and about the student's feeling they could have constructed the solution themselves. This is something else entirely. We're now talking not just about explaining an algorithm, but explaining the process through which the algorithm was devised.

When writing code, we are always encouraged to add comments explaining how the code works. When the code needs to be maintained later, it's helpful to have these comments rather than having to work out what the code is doing. But, if someone's maintaining the code, it seems likely that they may be needing to write some similar code of their own. Maybe they need to extend this piece of code, or write a similar method in another language. In such a case, "exposition" comments might be useful as well, talking about how the code came about, other options that were rejected, and so on.

In any case, it's interesting to think about, both in terms of mathematics and software development. If nothing else, I learned the word perspicuous, and got a bit of a laugh that it was the word chosen to explain about making ideas completely clear.

Owen at 2 months

Here are a couple recent photos of Owen. Yesterday was his two-month birthday!

13 is worth more than 14

In the NFL it is, anyway. Maybe.

I read a fair bit about sports, and in particular I have an interest in the statistics that people use to try to analyze them. One thing I've heard a few times as an example of a counter-intuitive stat is this: NFL teams scoring 13 points in a game win more often than teams scoring 14 points. I've recently come into possession of a database of game data, so I thought I'd have a look at this for myself.

My data contains all the regular-season and playoff games back to the 1978 season, so it's a pretty good sample size of about 7500 games. The first item to look at is the 13 vs. 14 thing, and sure enough:

13: 225-562-2 28.6%

14: 144-670-2 17.8%

There you have it - teams scoring 13 win significantly more often than teams scoring 14. Of course, the real question is what (if anything) this means. The most likely cause for this effect is the unusual way points are scored in football. Almost all scoring is through 3-point field goals and 7-point touchdowns. This means that 13 can really be thought of as 2 field goals and 1 touchdown, and 14 as 2 touchdowns. Maybe field-goal-heavy scores outperform touchdown-heavy scores in general. Let's see:

6: 18-307-0 5.5%

7: 16-621-2 2.7%

16: 249-256-0 49.3%

19: 160-129-0 55.4%

20: 543-432-2 55.7%

21: 308-405-0 43.2%

27: 568-176-0 76.3%

28: 328-152-2 68.3%

That certainly seems to support the FG vs. TD explanation, and in fact it's quite striking how poorly the multiples of 7 perform. 7 points wins 3% of games; it performs worse than 5, 6, 8, and 9 points. 14 wins 18%; it performs worse than 11, 12, and 13. 21 points is the highest score that loses more than half its games, and it performs worse than 16 (!). 28 does worse than 23, and so on.

The FG vs. TD explanation makes sense for a few reasons. First, the time one team is scoring is time that the other team isn't. 3 successful possessions, for a TD and 2 FGs, will generally take more time than 2 successful TD possessions, leaving the opponent with less time to score their own points. Second, teams that are trailing by a large amount won't try for field goals. That is, a team losing 20-7 will have to go for a touchdown, while a team losing 10-7 is more likely to take a field goal. Finally, there may be game conditions making certain games conducive to more field goals. For instance, a game with heavy snow or fog might reduce offense, causing both teams to score few TDs.

I must admit, though, that this effect doesn't last forever. Teams scoring 49 or 56 points have won 100% of their games since 1978. I guess the lesson is that if you're going to score touchdowns, you should try to score 7 or 8 of them, not just 2 :).

FileSystemWatcher

I recently came across the interesting FileSystemWatcher class in the .NET Framework. It's pretty cool; the class will watch a folder, and raise events when there are changes to the files in that folder. You can specify a filter (like "*.txt") to only watch for certain files, and react when files are created, deleted, and modified.

It's pretty easy to come up with possible uses for this class. Maybe you have an old process somewhere that produces data files at irregular intervals; you could watch the output folder, and immediately act whenever one of those files is added. You could create a kind of auto-publishing system; let your users know that any file they save in a certain folder will automatically be posted for them. You could set up a mechanism for communicating between processes.

This last one is something that I've seen done before in VFP. A timer is set up to repeatedly check a certain location for a "message" file from another process, and then the app can react accordingly. A FileSystemWatcher makes this kind of setup simple - just set properties specifying the files to look for, and the file system events to watch.

Implementing this is straightforward as well. Create an instance of the class:

FileSystemWatcher fsw = new FileSystemWatcher();

Set its properties, and hook up an event handler:

fsw.Path = "C:\\SomeFolder\\WatchFolder\\";
fsw.Filter = "*.txt";
fsw.NotifyFilter = NotifyFilters.LastWrite;
fsw.Changed += new FileSystemEventHandler(this.OnFileChange);

Write the handler to do the work:

private void OnFileChange(object source, FileSystemEventArgs e)
{
// ...
}

Interesting Numbers

There's a well-known (to mathematicians) story about a famous mathematician, Srinivasa Ramanujan. The story goes that Ramanujan was taking a taxi ride with another mathematician, Godfrey Hardy. Their taxi was number 1729, and Hardy commented that this was rather an uninteresting number. Ramanujan replied that it was in fact quite interesting, as it is the smallest whole number expressible as the sum of two cubes in two different ways.

Sure enough, it's true: 1729 = 10³ + 9³ = 12³ + 1³. This is the kind of thing mathematicians love, and Ramanujan is very well-respected, so this story is popular, and 1729 has even come to be known as the Hardy-Ramanujan number. Mathematicians also enjoy generalizing, so there's now a whole set of taxicab numbers, having to do with summing up powers like the cubes in this example.

There's another thing about this story that I like to think about - the notion of an interesting number. It seems an obvious and expected thing that some numbers are interesting and others aren't; there's even a Book of Curious and Interesting Numbers. This fact about 1729 makes it interesting (at least to me), and I can imagine some other number that has no similar interesting properties.

However, something odd happens if you try to actually find an uninteresting number. Let's just look at whole numbers, starting with zero. 0 is interesting because it is the additive identity, among other reasons. 1 is interesting because it's the multiplicative identity. 2 is the first prime number. 3 is the first odd prime number. 4 is 2². We can continue this way until we find our first uninteresting number. But wait! The first uninteresting number seems like an interesting property for our number to have, so it turns out to be interesting after all.

This seems like a bit of a trick, and maybe it is. There's some kind of paradox or weird self-reference at work here; our decision that a number is uninteresting is what makes it interesting. You can come up with more tricks like this without too much trouble. Here's another quick one, just to make the point: what is the smallest whole number that is not describable in twenty or fewer words?

With 20 words, you can describe a lot of numbers. One hundred; fifty million and three; three googol squared; Steve Wozniak's bank balance. However, there aren't an infinite number of english words, so there aren't an infinite number of 20-english-word combinations. That means there are more whole numbers than 20-word combinations, so some numbers are not describable in 20 words, and there must be a smallest one of these. But wait! The smallest whole number that is not describable in twenty or fewer words is only 13 words long, so this number is describable in less than 20 words after all.

Ok, it's another nice little trick. These are fun little diversions, and they've been known for a long time. They just seem like curiosities, though, without much real meaning or importance in more concrete matters. You wouldn't think that this idea of self-reference could undermine the foundation of all mathematical thought. It did, but that's a topic for a future post.

100% of all numbers contain a 3

Or: Infinity Weirdness, Part 1.

What percentage of all whole numbers contain at least one digit 3? It seems like a simple enough question. The simplest way to start trying to answer it is to have a look at some numbers, and do some counting.

Let's look at 1-digit numbers first. There are 10 of them: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9. Only one of these 10 contains a 3, so that's 10%.

Now, let's extend our list to the first 100 whole numbers: 0 through 99. We know that 10% of the numbers from 0 to 9 contain a 3. It's the same 10% with numbers from 10 to 19, because they are just the numbers from 0 to 9 with a 1 attached. Adding a 1 doesn't affect our count of numbers with 3s. Similarly, our count is 10% for 20-29, 40-49, 50-59, and so on. The interesting case is 30-39; obviously, all 100% of these numbers contain a 3. Taken together, our count looks like this:

0 - 9: 10%
10 - 19: 10%
20 - 29: 10%
30 - 39: 100%
40 - 49: 10%
...
90 - 99: 10%

We have 9 sets at 10%, and 1 set at 100%. Average this out, and we find that 19% of the numbers from 0 to 99 contain at least one 3.

The way this step from 1-digit numbers to 2-digit numbers worked gives us an insight into how this works generally. When we add a digit, we're adding each of the digits 0 - 9 to all of our existing numbers. 9 of these 10 digits (0-2, 4-9) have no effect on the count of 3s, and the last digit (3) creates a 100% count.

This means that if X is the fraction of n-digit numbers containing a 3, then the fraction of n+1-digit numbers containing a 3 is given by: 0.9X + 0.1. In more formal notation, this is a recursion where:

F_n+1 = 0.9F_n + 0.1
F₁ = 0.1

This can be expressed as a closed form in the following way (this can be shown with a small induction proof):

F_n = 1 - 0.9ⁿ

That's fine; the first few values of this are 10%, 19%, 27.1%. But we're interested in all the whole numbers, and the limit as n goes to infinity here is 100%. So we end up with an odd conclusion: 100% of all whole numbers contain a 3, even though not all whole numbers contain a 3. It's an important difference when dealing with infinite sets - 100% and all don't mean the same thing.

Stonefield in the Leader Post

Stonefield Software was recently the subject of an item in Regina's local paper, the Leader Post. The article is available online.

Blitz Tactics

Here are a few positions from recent blitz games. The solutions are in the comments.

Position #1. Black to move. Both knights are threatened; can Black play dxc3?



Position #2. White to move.



Position #3. White to move.



Position #4. Black to move.



Finally, from earlier in the same game. Black to move; what is his best?

Introducing Owen Scott Eisler

Owen Scott Eisler, born 4:58 am April 15, weighing 7lb 5oz.

American Idol Algebra

I ran into a little coincidence last week. There was a news story about an American Idol staffer claiming that the fix was in, and that the final four contestants had already been chosen. At the time, there were 11 contestants remaining on the show. Maybe the staffer just made this up - what are the odds that they could just guess the correct final four? The number we want is the number of ways to choose 4 people from a group of 11, and this is just the same calculation as was done in my Arby's Algebra post a few weeks ago. There are 330 possible final fours from a pool of 11 contestants, so the odds of guessing correctly are 1 in 330, or about 0.3%.

Of course, this quick calculation ignores the fact that not all contestants are equal, as some are known to be popular and so on.

SoftwareCEO Article

SoftwareCEO has an interesting article about the company I work for, Stonefield Software.

Formula Testing

Just testing a way to display formulas. If this works, it might make things clearer for posts like the Arby's one from a little while ago.

Happy Pi Day!

Today is March 14 - 3.14. Celebrate by having some pastry, or by measuring some circumferences.

Arby's Algebra

Arby's has recently had a commercial announcing their current deal, where you can choose 4 items from a list of 8 choices. They mention that this gives you 330 different options. Counting up this total is a nice example of combinatorics, so let's see how to figure out the total value.

Before getting into the math, it's helpful to assign a number to each of the eight menu choices. Rather than talking about an order as sandwich, fries, drink, apple turnover, it's simpler to call this 1,2,3,4. In the items below, when something like item 1 is mentioned, that just means I'd rather type "1" than "roast beef sandwich".

There are some approaches that are reasonable, but that don't give the right answer. First, you might consider that you have 8 choices for each of your 4 items, and simply count up 8*8*8*8, which gives 4096. This gives the total number of four-digit numbers where all the digits are from 1 to 8. The problem is that this double-counts some orders: the orders 1,1,2,3, 1,2,3,1, 3,2,1,1 are all counted as separate orders, when really they are the same.

Another approach is the 8 choose 4 method. The logic of this is that we have 8 items and we will choose 4 of them. There are 8 choices for the first item; once that's selected, there are 7 remaining choices for the next item, 6 for the third item, and 5 for the last. The total is 8*7*6*5 =1680. There are two problems with this. The first is that we're again double-counting some orders; this method will include both 1,2,3,4 and 4,3,2,1 as separate orders. We could solve that by dividing out these repetitions, but the second problem is worse: we're missing some orders. This method doesn't include repeating one item more than once. An order like 1,1,2,3 or 4,4,4,4 (for someone who loves apple turnovers) is never counted.

The correct way to look at this is as a "balls in buckets" problem (technically, combinations with repetition). The 8 menu items are the buckets, and we have 4 balls (choices in our order) to place in these buckets. If we place 2 balls in bucket 1, a ball in bucket 2, and a ball in bucket 3, that's an order of 1,1,2,3. All balls in bucket 4 is an order of 4,4,4,4. Now, here's the trick:

We can picture this as 7 dividers separating our 4 items into the 8 buckets. Visually, this means we translate our orders this way:

Order 1,2,3,4: o/o/o/o////
Order 4,4,4,4: ///oooo////
Order 2,4,5,8: /o//o/o///o

With the orders written this way, using the o and / characters, what we want to know is: how many different ways can we arrange 7 /s and 4 os? For this, we want an 11 choose 4 calculation. We have 11 total characters, and we need to choose 4 of them to be o's.

To start this calculation, we use 11*10*9*8, just like in the 8 choose 4 example above. As mentioned there, this has the problem of double-counting; it counts placing os in spots 1,2,3,4 as well as placing them in spots 4,3,2,1. This means that to get our final answer, we need to divide out these double-counts.

Luckily, that's pretty straightforward. The number of double-counts is just the number of ways to rearrange four numbers, and by the same logic we know this is 4*3*2*1. The final calculation is:

11*10*9*8 / 4*3*2*1
= 7920 / 24
= 330

Wikipedia link: Combinations

Let Me Google That For You

I am easily amused. Luckily, in this age of the internet, there are innumerable stupid little diversions available for us. The following example is great, because it also helps address the fact that I am easily annoyed as well. One little annoyance that's common enough is people taking the time to email or post a simple question, when it would take them less time to Google the answer themselves. Now, there's a site to help with such situations.

For example, say someone posts to a message board, asking for something like a listing of all the past champions of the Masters golf tournament. You can just reply, with a link like so:

Here you go.

3/3/09

Happy Square Root Day! Make sure you enjoy it; the next one's seven years away.

Strangest Bug Ever

I recently came across an extremely odd bug in Visual Studio 2003, which I've since learned is known as the Haunted Keyboard bug. I was editing some code, and suddenly a few of my keys stopped responding. Pressing enter, tab, or the arrow keys did nothing, while the normal letter keys worked normally. Very strange. I actually physically checked the keyboard to see if something was jamming or sticking to the keys, before opening Notepad and verifying that they worked fine there.

The explanation is that this bug allows one of the docked Visual Studio windows, such as the Toolbox, to grab focus for command keystrokes, but not others. The interesting thing is that for different versions of VS there are different fixes - in one case you can simply select the Toolbox and then set focus back to the code editor, and in another you need to reset all settings. Also you may need to use a fixed-pitch font, for some reason. Interestingly I could not find any mention of this happening in VS2003, where I was getting it, but only in VS2005. There were also claims that this issue was resolved for the release version of VS2005, but also postings of users still experiencing it there.

In any case, I was glad to find that this was a known issue. It was one of the few times where I've wondered if I was literally hallucinating the behaviour I was seeing. I'm happy I wasn't, partly because I don't particularly want to hallucinate, and partly because Oh noes my enter key doesn't work! is a pretty lame hallucination, even for me :).

Zugzwang

There was an interesting position in a recent game of mine. I'm playing black.

1.e4 c5 2.Nf3 Nc6 3.Bb5 e6 4.O-O Nge7 5.c3 a6 6.Ba4 b5 7.Bc2 Bb7 8.Re1 Rc8 9.a4 Ng6 10.axb5 axb5 11.Na3 b4?!

11...Ra8 is probably better.

12.Nb5 Be7 13.e5 O-O 14.d4 Qb6 15.Bd3 c4!? 16.Bxc4 Nxd4 17.Qxd4 Qxd4 18.cxd4 Rxc4 19.Bg5 Bxf3 20.Bxe7 Nxe7 21.gxf3 Nf5?

21...Rc2

22.Rec1 Rxc1 23.Rxc1 g5 24.Rc4 Nh4

24...Rb8!

25.Rxb4 Nxf3+ 26.Kg2 Nh4+ 27.Kg3 Nf5+ 28.Kg4 f6 29.Rc4 fxe5 30.dxe5 Rb8 31.Rc5 h6 32.b3 Kg7 33.Kh5 Kh7


This is the position, although most of what I'll say about it applies to the position a few moves later, after 35...Kg6 as well.

When I reached this position, I thought it was simply fairly equal. I don't mean a dead draw, since there are still some things to play with: White has a passed pawn, but his pawns are a bit split up; Black has a backwards d-pawn, and a nice knight on f5. However, looking at the position a bit more closely, White has a problem. He has no moves.

Let's go through the possibilities. White's knight can't move without dropping the b-pawn. White's rook can't move without dropping the knight. White does have some pawn moves, but not many, and none that really change the situation. b4 doesn't do much, and actually introduces a tactic once the king goes back to g4; f3 does nothing; h3 does nothing (except block a square the king might need). The king can move back to g4, but that allows the black king into g6, when White still has the same problems, and the added problem of Black pushing the kingside pawns.

Black, on the other hand, has as many waiting moves as needed, just by moving the king between g7 and h7. As a simple variation, if White tries to delay, Black can just wait: 34.f3 Kg7 35.b4 Kh7 36.h3 Kg7 and White is stuck, and probably has to throw away the h-pawn with 37.h4 Nxh4 38.f4 Nf3.

In the game, White retreated the king before using up all the pawn moves, and I was able to press forward for a win.

34.f3 Kg7 35.Kg4 Kg6

This is where the tactic against b4 appears. 36.b4 Nd4! and Black wins the b-pawn after 37.Nxd4 Rxb4, pinning the knight.

36.Kh3 h5 37.Kg2 Ra8 38.Rc7 Ra2+ 39.Kg1 Nh4 40.Rc3 Kh5 41.Re3 Kf4 42.Re4+ Kxf3 43.Re1 Kf4 44.Nd4 Rb2 45.Re2 Rb1+ 46.Kf2 Rh1 47.b4 Rxh2+ 48.Ke1 Rxe2+ 49.Kxe2 Kxe5 50.b5 Nf5 51.Nf3+ Kd6 52.Nxg5 Kc5 53.Kf3 Kxb5 54.Ke4 Kc4 55.Ke5 Kc3 56.Kf6 Ke3 57.Ke5 h4 0-1

Another Game

Here is another game, with some notes. This was a really interesting game, and I think I mostly played well.

1.e4 e6 2.d3 d5 3.Nd2 Nf6 4.Ngf3 c5 5.g3 Nc6 6.Bg2 Be7 7.O-O

This is a normal position in the King's Indian Attack against the French. Black commonly plays 7...b6 or 7...O-O here. However, my opponent played:

7...b5

Black often plays b5 at some point against the KIA; a common continuation is 7...O-O 8.Re1 b5. I'm not sure Black gains anything by playing this right away. In the game, he ends up castling soon enough, and for the next few moves there are some issues with the fact that he's opened the long diagonal.

8.exd5 exd5 9.c4 bxc4 10.dxc4 O-O

Here Black would consider 10...d4, if not for the fact that 11.Nxd4 is possible. Instead, Black takes a couple moves to clean things up, but ends up with some slightly weak queenside pawns.

During the game I thought that d4 was just impossible, but the computer doesn't think it's too bad. It gives variations like 10...d4 11.Nxd4 Nxd4 12.Bxa8 Bg4 13.Qa4+ Kf8 14.Qxa7 Ne2+ 15. Kh1 Nxc1 16.Raxc1 Qxd2. White has a rook and two pawns for a bishop and knight.

11.b3 Rb8 12.cxd5 Nxd5 13.Bb2 Bf6 14.Bxf6 Nxf6 15.Qc2

The opening is finishing up, and I think White has a good position. There is some pressure on the c5 pawn, and Black still needs to develop a bit.

15...Qa5 16.Ne4 Nxe4 17.Qxe4 Bb7

Black's king is looking a bit lonely.

18.Ng5 g6 19.Qh4 h5 20.Ne4 Nd4?!


This allows me to win the exchange, with a kind of a serial fork. Nf6 will fork the king and the d7 square, where it can fork the two rooks. 20...Ne5 instead would cover d7 and prevent this.

21.Nf6+ Kg7 22.Nd7 Bxg2 23.Kxg2 Qa6 24.Nxf8 Kxf8

White is up the exchange, but I found it quite hard to make further progress.

25.Qe4 Re8 26.Qd5 Ne6 27.Rfd1 Qc8 28.Qd7 Qa8+ 29.Qd5 Qc8 30.Rac1 a5 31.Rc4 Kg8 32.Rd3 Qa6 33.Rdc3 Rd8 34.Qe5 Qb7+ 35.Qe4 Qb6 36.Rd3 Nd4 37.Qe7 Rd5 38.Qe8+ Kg7 39.Qe4 Rf5

Fifteen moves since winning the exchange, and not a lot has changed. Maybe there is a way to continue to improve the position, but instead I played:

40.Rdxd4?!

Now I have a pawn-up rook ending. Even if this turns out to be objectively a good move, my thought process producing it was not so good. After going up the exchange, I was a bit frustrated at not being able to force more progress. Also, I've noticed in some of my games that I can be too eager for clarity in the position.

40...cxd4 41.Qxd4+ Qxd4 42.Rxd4 g5


Here is the rook ending. Another issue with my going into this is that I didn't make myself calculate it as deeply as I should have. Originally I thought I had a straightforward plan of trading the a pawns and putting my rook behind my remaining passed b-pawn. This ends up letting his king get quite active, with something like: 43.Rd2 Kf6 44.a3 Rc5 45.b4 axb4 46.axb4 Rb5 47.Rb2 Ke5 48.Kf3 Kd4. Instead, I decided to bring my king into the game.

43.Kf1 Re5 44.Rd2 g4 45.Re2 Rc5 46.Ke1 Kf6 47.Kd2 Kf5 48.Kd3 Rc1 49.Rc2!

My king is out, and will be able to get to his a-pawn. Black can win my h-pawn, and it looks like there may be some threats there to win some more pawns, or force one through, but I think that White can cover everything.

49...Rd1+ 50.Kc4 Rh1

Another approach for Black is to try to defend the a-pawn. 50...Ke6 51.Kb5 Rd5+ 52.Kb6 and Black is tied up. The rook can't leave the defense of the pawn, and White should be able to continue by trading the a-pawns and pushing the remaining passed pawn.

51.Kb5 Rxh2 52.Kxa5 h4 53.gxh4 g3

This would be very strong, due to the unprotected white rook, if not for the check White has available. As it is, White is winning, though not with too much room to spare.

54.Rc5+ Kg4 55.Rc4+ Kf3 56.fxg3 Rxa2+ 57.Ra4 Rh2 58.b4

58.Rf4+! is given by the computer, when 58...Kxg3 59.Rxf7 is a tablebase win.

58...Kxg3 59.b5 f5 60.b6 Rb2 61.Rb4 Ra2+ 62.Kb5 Ra8 63.b7 Rb8 64.Ka6 f4 65.Ka7 1-0

Black resigned, although even here, a little care is still required. After 65...Rxb7 White has to capture with the king, as 66.Rxb7? Kxh4 is a draw.

Keeping You Secure

VeriSign is a company which provides, among other things, digital signing certificates. These certificates provide a mechanism for website users to authenticate that the data they're receiving is legitimate. Of course, in order for this system to work, when you buy one of these certificates from them, VeriSign must confirm that you really are who you say you are. They can't very well issue you a signing certificate for XYZ Inc. if they can't prove you're from that company.

VeriSign has an information page which includes a listing of some of the items they will accept for authenticating your organization. One such item is:

Fictitious Name Statement

Calvin and Hobbes vs. the Bailout

Bill Watterson is a smart guy, I think.


From Greg Mankiw's Blog.

30% of all numbers start with 1

Well, not exactly. However, it is true that in certain common types of data, numbers beginning with 1 appear the most frequently, making up about 30% of the values. Numbers beginning with 2 appear slightly less frequently, about 18% of the time. Each successive digit appears with a lower frequency, until 9 shows up as the leading digit in less than 5% of the values. This property is called Benford's Law, after physicist Frank Benford.

This is quite a counter-intuitive thing. Why should there be more bank balances starting with a 1 than with a 9? Six times more of them, in fact. Why should this be true for the lengths of all the rivers in the world? The technical explanation for this is that these types of real world values are distributed logarithmically. For a more intuitive explanation, an example is probably more helpful.

Let's say you invest $100 in an account that pays 10% annually. This means that your investment will double every 7.3 years. The investment will reach $200 after 7.3 years, so for that entire first 7.3 years the investment's value began with a 1. Now, it will take another 7.3 years to double again. However, this time it's doubling to $400, not $300. The investment was valued in the $100s for the same amount of time it was valued in the $200s and $300s. The point is that the investment is growing at a rate proportional to its own size. It's compounding. The investment only spends 4 years or so in the $200s, 3 years in the $300s, and so on, finally breezing through the $900s in just over a year. Then this repeats for all the 4-digit values: 7.3 years in the $1000s, 4 in the $2000s, etc.

The reason I've been thinking about this topic recently is that it's been mentioned in relation to Bernie Madoff's ponzi scheme. Benford's Law is a good tool for detecting fraudulent data, because people faking such data often don't take it into account. Apparently, Madoff was sophisticated enough to generate numbers that met Benford's Law reasonably well.

As a quick real-world test of this, I thought I'd check the sizes of all the files on my computer's hard drive. The results seem to fit the Law's prediction quite well:

Digit	# Files	% Files	Benford
1	48,295	28.6%	30.1%
2	32,893	19.5%	17.6%
3	23,297	13.8%	12.5%
4	15,925	9.4%	9.7%
5	12,612	7.5%	7.9%
6	10,665	6.3%	6.7%
7	8,438	5.0%	5.8%
8	9,972	5.9%	5.1%
9	6,498	3.9%	4.6%

Wikipedia Link: Benford's Law

New Car

2009 Rogue!

Tactic #2

Here's a fairly complicated tactic, from computer analysis of a recent game of mine. Black to move.

Solution in the comments.

Most Obscure Function

I was looking back at some old Visual FoxPro code from a few years ago, and came across this line:

THIS.nShipAngle = RTOD(ATN2( .nShipY-tY, tX-.nShipX )) % 360

I figure that this ATN2 is the most obscure VFP function I've found a use for. It returns the arc tangent for a coordinate location, which is probably not a requirement that comes up too often in database application development :). However, when I needed it, there it was.

For interest's sake, the context of this is a clone of the video game Asteroids that I wrote. This is part of the code that aims the player's ship at the mouse pointer.

Super Bowl

Steelers 27 - Cardinals 20.

NFL Win Probability Calculator

Two things I've been interested in for a long time are NFL football and mathematics. This makes a site like Advanced NFL Stats right up my alley. In particular I like the Win Probability Calculator that was recently posted there. You enter a game state, including the score, field position, and time remaining, and the system produces the probability of each team winning the game. I think this could be a great tool for arguments about going for it on 4th down, or for attempting 2-point conversions.

Tactic

Here is a tactic, from another Facebook game. Black to move. (The board is shown from White's perspective; White pawns are moving up the board.)


I'll post the solution later in the comments.

First Post!

For my first real post, here's a chess game from Facebook, with some light comments.

Dixon, G - Eisler, R

1.Nf3 d5 2.d4 Nf6 3.c4 dxc4 4.Nc3 a6 5.e4 b5 6.Be2 c6 7.O-O Bb7 8.Bg5 Be7 9.Qc2 Nbd7 10.Rad1 O-O 11.d5 exd5 12.exd5 Re8 13.Rfe1 h6 14.Bh4

So far I think I have done well through the opening, but now I miscalculate fairly badly to end up in a worse position.

14...Nxd5?! 15.Nxd5 Bxh4 16.Nxh4 Qxh4 17.Nxc7 Qg5 18.g3 Nc5?

This had the idea of going to d3 with the knight, but it doesn't really work. 18...Qc5 is better, and the computer found some interesting ideas with 18...Ne5. For example, 18...Ne5 19.Nxe8 Rxe8 20.f4 Qf6 21.fxe5 Qb6+ 22.Kf1 Qc6 23.Kf2 Qb6+ 24.Kf1 is a draw.

19.Nxe8 Rxe8 20.Bxc4! Rxe1+ 21.Rxe1 Be4 22.Bxf7+

This creates a queen ending with White up a pawn; 22.Rxe4 Nxe4 23.Qxe4 bxc4 24.Qxc4 is a similar ending, with a slightly different pawn structure. This may have been a better option for White, since his extra pawn is on the queenside in this case. He may have been able to create a passed pawn there, which can be quite strong in a queen ending.

22...Kxf7 23.Rxe4 Nxe4 24.Qxe4 Qc1+ 25.Kg2 Qxb2 26.Qb7+ Kf8 27.Qxa6 Qe5 28.Qa8+ Ke7 29.h4 Kf6 30.Qd8+ Kf7 31.Qd2 Qe4+ 32.f3 Qe5 33.Qf4+


White is trying to trade down into a pawn-up king and pawn endgame, but this particular queen trade breaks up his pawn structure and causes him some big problems.

33...Qxf4 34.gxf4 h5!

White won't be able to protect his f4 pawn. Black's king will get to f5, and White will not be able to stay on e3 or g3, since Black has spare pawn moves available. So, White tries taking the b-pawn and creating a pawn race.

35.Kf2 Kf6 36.Ke3 Kf5 37.Kd4 Kxf4 38.Kc5 g5! 39.hxg5 Kxg5 40.Kxb5 h4 41.a4 h3 0-1

Black will win the pawn race.