Duplication

Its been a while since ive encountered two weird C++ bugs in one week.

The first was on OSX, where the compiler generated two copies of the same class which both contained a union but with different layouts.


template <typename _Kind_>
class Object
{
public:

enum ObjectType
{
TYPE_VOID,
TYPE_KIND,
};

union ObjectData;
{
void * Void;
_Kind_ * Kind;
};

ObjectType Type;
ObjectData Data;

void * operator & () const
{
switch(Type)
{
case TYPE_VOID: return (void*)Data.Void;
scase TYPE_KIND: return (void*)Data.Kind;
}

return 0;
}
};


The resulting bug was exposed when a non const version of the object was used to call the const operator.


int value = 8;

Object<int> obj;
obj.Type = Object<int>::TYPE_VOID;
obj.Data.Kind = &value;

// This calls the const operator which may return an invalid
// memory location. When dereferenced this will cause an error.

int * result = &obj;

if (*result == 4)
{
// ...
}


Strange but true, obviously the code producing the bug was a little more complicated but semantically exactly the same as that above.

The second bug was not quite so serious, but still very confusing as it relates to one of those obscure C++ standard "undefined behaviors". Sometimes i wish the standards body would realise that the language may actually benefit from having some consistency and predictability.

This time the code was to perform some decoding from ascii to binary hex values on Windows and used a lookup table to convert 2 ascii bytes into 1 binary byte.


char entry[] = {0x01,0x02,0x03,0x04,0x05};
int table[] = {0x01,0x02,0x03,0x04,0x05};

int decode[1];
decode[0] = (table[*entry++]<<4)|(table[*entry++]);



However the result was not what you would expect. The assembly generated looked something like this. With the result being that the two post increments did not
occur until after the assignment. Weird... ?


009603A7 mov eax,dword ptr [entry]
009603AA movsx ecx,byte ptr [eax]
009603AD movsx edx,byte ptr table (0BC4A60h)[ecx]
009603B4 shl edx,4
009603B7 mov eax,dword ptr [entry]
009603BA movsx ecx,byte ptr [eax]
009603BD movsx eax,byte ptr table (0BC4A60h)[ecx]
009603C4 or edx,eax
009603C6 mov byte ptr [decode],dl
009603C9 mov ecx,dword ptr [entry]
009603CC add ecx,1
009603CF mov dword ptr [entry],ecx
009603D2 mov edx,dword ptr [entry]
009603D5 add edx,1
009603D8 mov dword ptr [entry],edx


But it turns out there is an obscure C++ feature (see section 5, paragraph 4 of the ISO C++ standard) whereby you are not allowed to modify the same variable twice in a single expression. Doing so results in the above "undefined behavior". Great.

Why this needs to be so i dont really understand.

So there you go, two annoying bugs in one week. Both required some pretty heavy problem solving to see.

C++ is a hard language to love.

Privateering

Nothing divides programmers more than coding style, in particular the representation of private member variables.

Here are some of the options;



class Class
{
    m_member
    m_Member

    _member
    _Member

    member
    Member
};




The problem is, if you are like me you don't particularly like prefixes like m_ or _ and the problem with not using a prefix is that it can conflict with method names.

Using prefixes unfortunately can create bad code since it encourages programmers not to think about naming or the number of members in a class.

The solution can be to use a lower case member name and an upper case method name, but then there is no distinction between local variables and member variables within a method body.

Another less common solution which i use sometimes, since i always prefer to use no prefixes and uppercase member names just like methods, is to wrap my private members in a struct to remove the naiming conflict but keep the cleanliness and readability.

class Class
{

    struct PrivateMembers
    {
        Member;
    }

    PrivateMembers Private;

    void Member()
    {
        Private.Member; 
    }
};

Shifting

Dos Date Time
Two's Compliment

I recently encountered this strange but simple bug in some code that i had written to convert from posix standard to dos standard time. It was strange becuase it only shows up when you use dates and times which are earlier than a certain magic number.

Dos uses a strange bit packed representation of date and time in two 16 bit words. Within each word the various values occupy a set number of bits as follows.


Date Format

0-4 Day of the month (1–31)
5-8 Month (1 = January, 2 = February, and so on)
9-15 Year offset from 1980 (add 1980 to get actual year)

Time Format
0-4 Second divided by 2
5-10 Minute (0–59)
11-15 Hour (0–23 on a 24-hour clock)


So assuming you have some simple integer representation of time and date, lets call it a Calendar you might use the following code to convert to and from the Dos format.

// Conversion to dos date and time
Calendar calendar;
short date = ((calendar.Year - 1980)<<9)
  + ((calendar.Month)<<5) + calendar.Day;
short time = ((calendar.Hour)<<11) 
  + ((calendar.Minute)<<5) + (calendar.Second/2);

// Conversion from dos date and time
Calendar calendar;
calendar.Year = ((date>>9)&0x007F)+1980; // 7 bits
calendar.Month = ((date>>5)&0x000F);  // 4 bits
calendar.Day = (date&0x001F);   // 5 bits

calendar.Hour = ((time>>11)&0x001F);  // 5 bits
calendar.Minute = ((time>>5)&0x003F);  // 6 bits
calendar.Second = (time&0x001F)*2;  // 5 bits

So wheres the bug ?

Well, the bug occurs when you use a year earlier than 1980, for example 1970. What happens is that thanks to the subtraction of 1980 from the year you end up with a negative number which is left shifted 9 bits into the 16 bit word.

Now usually when you do bit packing, its standard practice to mask the bits as you extract them by using an binary and after the right shift operation. This ensures that the value you get is isolated from the other bits, especially those further to the left since usually you right shift until zero.

In this case though the year is different from the other values in the date word. Becuase the year occupies the left most bits, its two's compliment representation as a negative number is preserved all the way through the right shift operation until the binary and, where it is suddenly forced into being a positive integer with the same 7 bits, changing it from a small negative to a large positive number.

The solution is to remove the binary and mask on the year. For all other values the mask is still necessary, but due to the weird although perhaps purposeful design of the dos date and time format, the year is special.

Released

Finally after many sleepless nights, numerous relationships, two continents, and a lot of cold weather i have released some of my work into the wild under the GNU General Public License.

Its a C++ framework called Reason, and represents about 40% of the code that ive produced in ecent years. I hope you enjoy reading it, and maybe someone will even find it useful :)

http://reasoning.info/examples.htm

Axioms

I was fumbling through some old code this evening after discovering that my original interpretaion of the slightly skewed BNF grammar used in FRC2396 was incorrect. Obviously despite creating the standard Sir Tim Burners Lee hadn't quite got the hang of hyperlinking yet, and so it is generally left up to the reader to discover the reference to RFC822 in the third paragraph of section 1.6, and hence forth seek out the correct interpretation.

Speaking of hyperlinking, i found an amusing anecdote in my comments which i thought might be worth sharing. It originates from the W3C axioms of web architecture, and pretty much states that the internet in its current form is a complete abomination.


All the same, a word of caution is appropriate about the indiscriminate or deliberately misleading abuse of the identity of the object referred to by a URI. A web server is often in a position to know a lot of context about a request. This can include for example, the person who is asking, the document they were reading last from which they followed the link. It is possible to use this information to ramatically change the content of the document referred to. This undermines the concept of identity and of reference in general. To do that without making it clear is misleading both to anyone who quotes the URI of a page or who follows the link.

Unless it is clearly indicated on the page (or using a future protocol) , to return differing information for the same URI must be considered a form of deception. It also of course messes up caches.


Reflect on this next time you create some dynamic content...

Superstition

Sometimes, just sometimes the compiler is actually right.

The problem is, not always. Like every nascent computer user sometimes even seasoned programmers arbitrarily distrust the machine.

Technofear.n - The result of seemingly random 
consequences caused by the actions of inept users 
leeding to a deep sense of confusion and distrust 
for technology. i.e. "programming the vcr"

You see, as a programmer, even more so than a user it is essential to be able to trust the tools that you use to produce the results that you expect. The programmer must have faith that the compiler tells the truth in all matters and all things.

For if the compiler lies, then the programmer is sure to spend many countless frustrating hours chasing bugs which don't exists. And eventually their confidence will be eroded to the point that they will also waste countless hours swearing at the compiler for producing an error, which in fact is genuine.

Unfortunately its a bit of a boy who cried wolf scenario, and the horrible truth and shame about the software industry, is that even today, all compilers lie.

Anyone who has ever misplaced a curly brace or two and received a cascade of endless random errors from their compiler knows this shameful fact of life.

So then, it is left to the programmer to develop their own intuitive understanding of the particular tools and compilers that they use, to the point that they know when an error is and error and when the machine just needs a good kick. This truly is the art of programming, and perhaps more than you might think it can separate the good programmers from the great programmers, and at the very least, the frustrated from the productive.

Ive just fixed a rather good example of such an error. And fortunately for me, i chose wisely in trusting the compiler in what seemed like a perfect opportunity not too. You see, my program was operating correctly in debug, but failing in release.

An obvious problem you may suggest, but not quite. You see, if i slowed the program down in release, it worked correctly. Ahhh! i hear you say, "clearly youve got a race condition between one or more threads". But alas, there is only a single thread and the timing is very subtle.

It sure did look like one of those occasions where one may be tempted to start looking under the rug for a wolf. But i held fast and debugged the code into the long hours of the night. And after much toiling, and tracing, and printing as there are only limited ways one can effectively debug release code, i started to narrow the trail.

After two days of debugging i found the problem.

It was caused by two objects being compared by memory addresses. In the world of debug code, memory addresses are consecutive manufactured things, contrived to make the task of debugging easier for the programmer. But in release code, memory addresses are random highly variable things. So in debug, object A was less than object B, but in release where the memory was different, object B was less than object A.

At its root was an innocuous problem, but it was burried deep in thousands of lines of code. Knowing when to trust and when to distrust your tools is critical, becuase sometimes...

The compiler is actually right.

Epoch

We are Microsoft, we will store dates with a different epoch and a different granularity to Unix.

An epoch of 01 Jan 1970 00:00:00 isn't good enough for us, no we want 01 Jan 1601, 00:00:00. There much better.

And whats this seconds rubbish, we want 100 nanosecond ticks thanks very much. And we will call it file time so as to really confuse you.

But its ok, well give you this really obvious conversion function so you can make all your code non standard.

#include 

void TimetToFileTime( time_t t, LPFILETIME pft )
{
    LONGLONG ll = Int32x32To64(t, 10000000) 
    + 116444736000000000;
    pft->dwLowDateTime = (DWORD) ll;
    pft->dwHighDateTime = ll >>32;
}

Did you catch that ?

116444736000000000 / 10  = 11644473600000000 (Microseconds)
11644473600000000 / 1000 = 11644473600000 (Milliseconds)
11644473600000 / 1000  = 11644473600 (Seconds)
11644473600 / 60  = 194074560 (Minutes)
194074560 / 60   = 3234576 (Hours)
3234576 /24   = 134774 (Days)
134774 / 365   = ~369 (Years)

Which is roughly the interval between the two epochs, give or take a few leap years and leap seconds.

Oh yea, and then in .NET we will change it all again, just to be cool. Now we want the ultimate epoch - midnight on the 01 Jan A.D. Beat that Richard Stallman.

Multiplicity

Finally, someone in the C++ community that realises how far the language has fallen behind becuase its missing a decent library.

"I'm going to speculate now, because I haven't been watching Java and I've just got my feet a little bit wet in .NET. But I would say that while the C++ community was focusing on templates, the STL, and exceptions—oddly enough the three are wrapped up together pretty closely—what they were not doing was component-based development. For example, there is no huge collection of class libraries for C++. The standard library for C++ is pretty impoverished. In the meantime, the rest of the world was busy creating huge class libraries that let you write all kinds of really neat applications without having to write very much code. Certainly Java is famous for its libraries. .NET has a huge number of libraries."
- Scott Meyers

Abstract Base Classes

I was reading the above article becuase ive just been using multiple inheritance in my own library. The design forced me to use it and i wanted to validate some of my rational. Multiple inheritance allowed me to connect up the notion of streams with that of lower level IO like files and sockets. So whilst i have a File class and a Socket class i also have a Stream class and an associated FileStream and SocketStream, each of which derive multiply from the base class of that type and implement the Stream interface. The interesting part is that Stream implements two other interfaces, Reader and Writer. File and Socket also implement Reader and Writer which define some standard contracts for reading and writing bytes.

class Reader
{
    ...
    virtual int Read(char * data, int size)=0;
    virtual int Read(Writer & writer, int amount=0);
    ...
};

class Writer
{
    ...
    virtual int Write(char * data, int size)=0;
    virtual int Write(Reader & reader, int amount=0);
    ...
};

This is a very useful notion, it lets me have Stream's which i can use for efficient progressive reading, writing, and seeking, but i can also use these streams interact with their more primitive classes. For instance, to read from a SocketStream directly into a File i can do this:

SocketAddress socketAddress("www.google.com",80);
SocketStream socketStream;
socketStream.Connect(socketAddress);
socketStream.Write("GET / HTTP/1.0\r\n\r\n");

File file("/tmp/google.html");
file.Write(socketStream);

And now i have a local copy of the html page. It works becuase the Reader and Writer interfaces allow direct connections to one another. This provides interoperability between streams and other forms of IO when i just want to do a simple redirection. But i can still harness all the pipelining capability of streams when i need it.

The inheritance heirarchy of the FileStream class looks like this:

FileStream
[
    File
    [
        Filesystem
            [...]
        Reader
        Writer    
    ]    
    Stream    
    [
        Reader
        Writer
    ]
]

Of course the FileStream can do everything that a File can do, but it also blends with it Stream semantics. Multiple inheritance allows the common interfaces to be merged and the simple composition of two distinct usage patterns into a single class.

Its nice when an intuitive design just jumps out at you, its a rare example of a truly useful piece of multiple inheritance.