Mad Software

Console Resolutions in Ubuntu

2011-10-11T22:11:00.000-07:00

I recently (well, like a year ago) switched my personal server from Gentoo to Ubuntu server and have been generally pleased. However, when I tried putting the server in a closet and hooking up an old cheap LCD monitor to it, the text mode wouldn't come up right. It was all white and hard to read. I could figure out what was on the screen, but it was annoying every time I had to use the console for some reason or another. The problem was clearly that the wrong resolution or refresh rate or some other monitor setting was not being set correctly.

So I go online to find out how to switch the console resolution. Everything I can find talks about doing this by changing the kernel options. Great, I've done this before. I spent far too much time fiddling with vga=0xF00 on my old Matrox G400 trying to TV-Out to work. Just gotta figure out how to do it again.

No big deal right? Well, now Grub has exploded from a couple config files in /boot to several sets of config files in /etc and /boot. There's even a grub.d directory. I can see the Emacs syndrome coming on. It's just a bootloader folks. It doesn't need to be this complicated.

Anyway, I get this all figured out. Turns out that we don't use vga=791 or whatever anymore. That's deprecated. We now use set gfxpayload=640x480, but not on the kernel line. It goes somewhere in grub.conf. But we can't set it there because that gets overwritten by update-grub. Argh.

Oh! There's this file over here called /etc/default/grub that I change. But set gfxpayload isn't supported in that. /etc/default/grub looks like a bunch of environment variable definitions. Actually gfxpayload is supported, but it's not documented anywhere except in the bug report that added the feature. This is becoming not simple. It's called GRUB_GFXPAYLOAD_LINUX=640x480, by the way.

Ok, we're good now. Changed /etc/default/grub, ran update-grub. Reboot. Watching BIOS stuff, graphics card, IRQ list and Grub comes up. Yay! It's working. Alright, select the kernel and hit enter. We're booting. Uh oh, why is the mode changing again? Argh, it's still wrong. Apparently the mode change only works in Grub, then the kernel switches it to something else. Something defined in some config file I'm never going to find. Why are there so many posts about this if it only works in Grub? I don't really spend a lot of time in Grub. I'd think most folks would be ok if the display was a little glitchy in Grub. I'm missing something.

Should I post to a forum? No, I want to fix this now. If I don't fix it now, it'll just keep annoying me just enough to spend half an hour of my time trying to fix it every time I need to use the console, but then I'll give up and forget what I was trying to do in the first place. Then my wife will ask why it's taking me so long to fix that thing that I said would take five minutes. Uh, cause I'm not actually fixing that. I just say, "What was that? I couldn't hear you. Almost done dear!" I think she's on to me. She knows what this really means. I get on IRC.

Nice folks on #ubuntu, but not generally that helpful in certain situations. Nicer than the folks on #gentoo. I ask my question. Someone asks if I've done the obvious things. Yes, I've already done that. I almost decide to try to help some poor newbie with his problem. Nope, that will only delay things further. Tonight, guilt about being a better citizen doesn't make me do the right thing. Focus.

While in #ubuntu, someone posts something about completely unrelated, but pastes some part of their dmesg. Of course! Why do I always forget to check the logs? Checking dmesg. Uh, why is this "nouveau" thing changing my resolution to 640 by 350? 350? I don't think that's too standard. No wonder my cheap LCD is having issues. I'm surprised it's working at all. Now I'm on the right track.

Google "nouveau". Ok, this is better. Ahah! I just need to add some video=640x480 line to the kernel line. Good thing I now know about /etc/default/grub. Edit. update-grub. Reboot. Yay! It works.

Why are there fourteen different ways to deal with the console resolution settings? This is far more complicated than it should be. Now I remember why I could never get things working on my Matrox G400 just right. It's one of those terrible places where it's just too complicated, but no one is really using it. There are about two people in the world who know the answer to your question, and they don't get on IRC.

Why am I the one why gets the issues that no one else in web land ever gets?

Using malloc Safely in C++

2011-05-06T16:23:00.000-07:00

For many of us C++ programmers, a key advantage of using the language is the compatibility with C. For the most part, this interface works well. However, there are some places where this interface isn't so easy to deal with. Take memory allocation for example. In C, one typically uses malloc (or some equivalent for the platform) to allocate memory and free to release it. In C++, we tend to use the built-in new and delete operators to accomplish the same thing.

When interacting with C libraries, it can sometimes be impossible to avoid using malloc and free. Libraries often assume that any memory that it allocates and hands off to the user will be freed using free. Since you can't tell what sort of memory a particular pointer points to, it's easy to make mistakes when deallocating them. Using mismatched allocators/deallocators might cause memory leaks, crash your program and work just fine depending the platform, compiler, or phases of the moon.

The first thought that any self respecting C++ programmer will think of is RAII (Resource Acquisition Is Initialization). So they might write a class that automatically calls free when the pointer is no longer needed.

template<typename T>
class malloc_handler
{
public:
 malloc_handler() : _ptr(malloc(sizeof(T))) {}
 ~malloc_handler() { free(_ptr); }
 T* ptr() { return _ptr; }
private:
 T* _ptr;
};

void foo()
{
 malloc_handler<int> x();
 x.ptr();
}

This has a number of disadvantages including the lack of a copy constructor, the requirement of a new class and the lack of any convenient operator overloads to make it look more like a pointer. There's a much better way and one that doesn't require the programmer to roll their own solution. Use shared_ptr and its customer deleter feature.

shared_ptr<int> x((int*)malloc(sizeof(int)), free);

This feature is probably something that the creators of shared_ptr already envisioned, but I don't generally see it discussed. Now, the above code is a bit ugly, so we can make it a little bit easier by rolling our own code.

template<typename T>
shared_ptr<T> shared_malloc()
{
 shared_ptr<T> rval((T*)malloc(sizeof(T)), free);
 return rval;
}

shared_ptr<int> x = shared_malloc<int>();

We can create a safe version of strdup:

shared_ptr<char> shared_strdup(const char* str)
{
 shared_ptr<char> rval(strdup(str), free);
 return rval;
}

shared_ptr<char> s = shared_strdup("Hello World!\n");

Science and Religion

2011-02-22T09:32:00.000-08:00

Recently the following article appeared over at Cosmic Variance, a blog I follow. Theologians Lobby Successfully to Change Definition of Evolution Some theologians (Cornell's Huston Smith and Notre Dame's Alvin Plantinga specifically) asked that the following definition of evolution used by the National Association of Biology Teachers be modified to remove the words "unsupervised" and "impersonal".

The diversity of life on earth is the result of evolution: an unsupervised, impersonal, unpredictable and natural process of temporal descent with genetic modification that is affected by natural selection, chance, historical contingencies and changing environments.

I think it's a good idea. I believe in science, I believe in religion and I believe in evolution. But I have a hard time understanding why it’s so important to some scientists to make sure that not only do we not mention God in science, but we must specifically reject him outright. I agree that there is no evidence that God is there. If there were evidence, then it would be pretty easy to believe in God. I’ll accept that. I don’t expect you to believe in God. But since there is also no evidence that he doesn’t exist, then I expect science to leave the topic alone. Let the Atheists discuss the absence of God. Science is about evidence, not belief.

I believe in science because I believe that science is a valid and useful way to discover things about the world. It’s been astoundingly successful at doing so. I don’t feel I can reject it, nor do I want to. I benefit from it every day.

I believe in religion because I believe that religion is a valid and useful way to discover things about the world. I believe that spiritual experiences are real and I can reconcile them with the things I learn from science. It may not be in the way that some scientists may think I should, but I do it all the same.

I believe in evolution because science has shown that there’s little chance that it happened any other way. I’m not going to reject the evidence that is clear as day. However, just like a scientist cannot reject “uncomfortable” evidence when it contradicts a pet theory, I cannot reject my spiritual experiences. They are as real to me as any evidence that science can provide. Are they perhaps nothing more than my brain being affected by too much dopamine? Perhaps. And I may reject them if sufficient evidence is presented to me. But sufficient evidence has not been presented to me. So, until that time, I must reconcile the evidence with which I am presented.

Just because I see some conflicts between science and religion doesn’t mean that I reject either one. General Relativity and Quantum Mechanics have conflicts. I don’t just blindly throw one out. They both have some of the truth. One day, we’ll understand how they fit together. I’m okay with some conflicts. I know they’ll be resolved eventually.

But I do expect you to state conclusions based on good evidence. There is no evidence that God didn’t have some small hand in the evolutionary process. Perhaps evolution is the mechanism by which he created a diverse ecosystem on this planet. That’s what the theologians are asking for. The National Association of Biology Teachers is overstating what science knows about evolution.

The words unsupervised and impersonal are quite problematic. “Unsupervised” implies the absence of some intelligence involved. Well, what is intelligence? Once humans showed up did evolution stop? We can discuss a number of situations where humans had a clear hand in the evolutionary process. What about other possibly intelligent animals? How much intelligence do you need before you’re a “supervisor”? And “impersonal”, which would mean that no person was involved. Well, since humans didn’t show up until pretty late in the game, that’s sorta obvious, but has the same problems as “unsupervised”.

I’m really getting tired of two things in the scientific community. One, the alliance with the atheists which some in science seem to think is the only way it can be. Science and religion can coexist peacefully you know. And two, the accusations of stupidity at anyone who doesn’t accept everything a scientist says without question. I understand you’ve studied your field in more depth than anyone else, but that doesn’t alleviate the need to explain your reasoning to those you’re speaking with.

When I have a scientific discussion, I don’t bring my religion into it since it’s irrelevant to everyone else. Some scientists are atheists and they should do the same. Scientific statements should be scientific. There’s no place for statements that God doesn't exist when there’s no evidence for that statement. If we want the world to accept evolution, then please stop overstating the case for it. The evidence is convincing enough without saying things we don’t really know.

"Is there any conflict between science and religion? There is no conflict in the mind of God, but often there is conflict in the minds of men." — Henry Eyring, developer of the Transition State Theory of Chemical Reactions

Calling a constructor or destructor without an object

2011-02-17T09:33:00.000-08:00

Now, I have no idea why you'd actually want to call constructors or destructors without an object, since they're really normally meant to be called with objects, but it's something interesting I thought I'd share. It turns out that constructors and destructors aren't really that special in C++. They're just regular functions like almost anything else and can be called that way. You just need to know how to do it.

I'll start with destructors, which may seem backwards, but doing this with destructors is actually easier than with constructors. It's not that much easier, but it makes a bit more sense. Let's start by calling a destructor with an object. We'll get to the no-object case next.

To call a destructor if you have an object, well, just call it:


struct A {
  A() { cout << "A()" << endl; }
  ~A() { cout << "~A()" << endl; }
};

void main()
{
  A a;
  a.~a(); // just call it like anything else
}

So how do we do this without an object? We need to think about what happens when any member function is called. When the compiler compiles a member function call, all it does differently from a normal function call is to add a pointer to the object as an extra parameter (at the begining) and call the function normally. Now certain compilers even use a different calling convention, but in the end, it's just a modified function call.

So can we get the compiler to make that function call, but not require a real object to pass in? Of course:


A* a = NULL;
a->~a();

or more compactly:


((A*)NULL)->~a();

Now, as long as nothing in the destructor actually tries to access the this pointer (by accessing data members for example), then all is well. The destructor is called and no object was ever created.

What is this useful for? I have no idea, but one thought that I had centered around one particularly special behavior of destructors. They automatically call the destructor of the parent class. Every other type of member function doesn't do this. Parent class versions of functions much be explicitly called. In some cases, this can be frustrating, especially in template classes where the parent class may be hard to determine.

In order to call a constructor without creating an object, we can use a special feature of C++ rarely seen outside of things like STL and embedded code, placement new. Placement new is just like new, except that it doesn't allocate memory for you. It leaves the memory allocation up to the programmer. Here's what it looks like normally.


void* ptr = malloc(sizeof(A));
A* a = new (ptr) A();

The syntax is a little funny, but that's okay. All we're doing there is calling the constructor member function using our own pointer (ptr) for the this pointer instead an automatically generated one.

So, using the same trick of using NULL instead of a valid pointer as we did with the destructors, we can do the same with constructors:


A* a = new ((void*) NULL) A();

Except it doesn't work. At least with gcc. The constructor never is called since the this pointer is NULL. Well, no worries, we're not trying to actually do anything with the this pointer, so we just need to pass in something that the compiler thinks is good.


A* a = new ((void*) 1) A();

Let's get rid of the assignment, since we're not going to use that pointer anyway.


new ((void*) 1) A();

And voila, we're calling constructors. No object needed.

One key point here that should be repeated is that none of this will work on destructors or constructors that actually try to dereference the this pointer in any way. The most obvious time this is done is when accessing data members. Another is calling virtual functions, since the virtual function table will need to be referenced. Also, any class hierarchy that includes virtual inheritance won't work since that mechanism uses the this pointer as part of knowing what destructor to call.

Emacs for Visual Studio users

2011-01-20T13:32:00.000-08:00

Now that I'm using Linux to do all my development, I'm finding that a few things that I got used to in Visual Studio can be a little more difficult in emacs and a terminal. However, I'm finding that emacs generally has all the features I'm looking for and more if I take the time to learn them. Here are a few features I used in Visual Studio for which I've found equivalents in emacs.

Find in Files

"Find in Files" is a particularly useful feature in Visual Studio. For a while I was just using grep in a terminal and manually jumping to files it found. Slow and annoying. M-x find-grep is just what I was looking for. It would be nice if I could more easily tell it not to search everything and just *.cpp and *.h files for example.

M-x find-grep-dired puts the found files in a dired buffer (something I need to learn more about) and lets you do things like query-replace on a marked file. I haven't used this but it seems much more powerful than Visual Studio's system.

Debugging

Until I'm convinced otherwise, I still think that Visual Studio's debugger is the best in the business. Qt Creator comes close but the rest of the interface there isn't so hot. Emacs with gdb is surprisingly good and much better than the last time I tried using it a few years ago.

One problem is that I can't set or even enable a breakpoint while the program is running. Sometimes this is really useful as you have to run some code that will hit the breakpoint a lot before the one time you're interested in. Sometimes, it's really a matter of clicking "enable" at just the right time. Conditions help here, but sometimes it's just easier to enable when you need to.

Saving breakpoints was something I didn't think I could do, but the "save breakpoints" command seems to overcome that. Not as clean as Visual Studio, as I have to "source" the resulting file when I restart.

One command that has promise is rbreak, which I can use to set a breakpoint on all functions which match some regex. I couldn't get it to work however. It seems to find all the functions, but when it tries to set the breakpoints, it can't find them and asks if I want to delay setting them. Bug.

File Navigation

I like "C-x b", which allows you to load existing buffers quickly. I think there was something in Visual Studio that did this, but I never used it. If anything, I don't think it had tab completion, which emacs does.

iPhone on a virtual machine

2011-01-13T11:20:00.000-08:00

Ever since I started my new job where I don't have to run Windows all the damn time, I haven't sync'ed my iPhone. So, when I tried to just back it up on Ubuntu, it totally messed up the iPod app. So I decided I wouldn't try that again. Anyway, now that VirtualBox supports USB passthrough, I figured it was time to try to get it working again. I created a Windows 7 virtual machine and installed iTunes, moved my library over, and started syncing. Some things that might make it easier on those trying to do this:

1. Make sure you're in the vboxusers group. Things worked for me without doing this, but I had problems, so just do it even if things work for you.

2. The iPhone occasionally will disconnect on certain types of operations and come up as a different device which won't get automatically captured by VirtualBox. iTunes then waits and waits for the iPhone, but never sees it so bad things happen. Watch for this and tell VirtualBox to capture the iPhone when it happens. I think the USB filter thing in VirtualBox helps here. This is especially important when updating to a new version of iOS. If you have problems, look up UDF mode.

Switching on Boolean Conditions and Flags

2011-01-04T09:14:00.001-08:00

I just posted an article over at CodeProject.com about what I call a boolean switch or switch flags (as the library I wrote is called). I've thought a lot about it over the years and implemented in a couple ways. It reduces the complexity of nested if statements. Basically, it allows for code like this:


switch_flags_2(a > b, c != d)
{
case flags_2(T,T):
    // only execute when "a > b" and "c != d"
    break;
case flags_2(F,X):
    // execute when "a > b" is false
    break;
}

Gnome Panel Applets

2010-12-03T14:47:00.000-08:00

Just learned about Gnome panel applets because I wanted to make one to track my time at work. FYI, the Gnome Panel Extensions project is dead, or it never was really alive in the first place. I asked about this in IRC to which the response was:

for C, this http://projects.gnome.org/ORBit2/appletstutorial.html is ancient, but I think it's still more or less up to date. Not much has happened on the applet api front since forever. Python is the other reasonable choice and http://library.gnome.org/devel/panel-applet/stable/

panel-applets can use d-bus now http://live.gnome.org/GnomeGoals/AppletsDbusMigration not that much of a difference though

good point

and of course there's the fact that if gnome-shell actually does become successful in 3.0, applets go the way of the dodo along with the panel

We'll see if I get around to writing one. It's too bad that the panel extensions project is dead because they had some pretty good documentation. A simple tutorial or some sample code would be nice...

Exceptions are Faster

2010-11-24T11:53:00.000-08:00

There's a lot of C++ hating around these days. This hate doesn't always come from the world of young bucks in love with Python. Seasoned C programmers like to hate on C++ for it's nice features. Because, as all good purtitans know, anything that feels good or makes life easier must be bad.

One particular C++ language feature that is generally avoided are exceptions. Even C++ programmers, who love exceptions for their ability to clean up ugly error handling code, are guilty of this. The general consensus seems to be that exceptions should only be used in parts of code that aren't time sensitive. Cache invalidation and all sorts of other badness are usually the reasons given that exceptions are slow.

The alternative in that situation is to use return codes. Yeah, sure they're ugly and use up that nice return value slot that could otherwise be used for something nice, but nothing is sacred when performance is required.

Well, I was thinking about that today and decided that this is all wrong.

The logic is this: In order to handle return codes properly, you must check them all the time. When you're writing time sensitive code, checking an error code that isn't going to be set 99.9% of the time is a big fast waste of time. In fact, looking at any flag that isn't set 99.9% of the time is really stupid. There's usually a better way to handle the logic. Spending the same amount of time for rare cases and common cases doesn't make sense. It's okay for the code that handles rare cases to be slow as long as the common case is as fast as possible. The overall speed won't be significantly affected by the rare case.

This is why exceptions are so great. With exceptions, you don't actually check anything most of the time. In other words, the common case of no error, takes no time at all. The compiler takes care of figuring out where exceptions should be caught at the points in the code where they're thrown. The way it does that is to simply "goto" the catch point. When using an error code, the calling code is checking for the error code every time. Yeah, it's slow because it's jumping all over the place. But so what? It only happens on rare occasions, which can be slow. If your code is designed such that errors are relatively rare occurances (as it should) then there is likely very little impact. Of course, if you're using exceptions for program logic, then well, you're doing it wrong.

All right, here's the proof that exceptions are faster than return codes. I created simple class that increments a value until it hits some maximum value. It is considered an error when it hits that maximum value. The return value version of this simply returns false when an error occurs, while the exception version throws an exception.

The return value version:

class inc

{

public:

inc(int m) : m_max(m), m_count(0) {}

bool go(int& x)

{

if (m_count > m_max)

return false;

x = ++m_count;

return true;

}

private:

int m_count, m_max;

};

int main()

{

int count = 1000000000;

inc f(count);

int x;

while (1)

{

if (!f.go(x))

{

printf("error\n");

return 1;

}

The exception version:

struct my_exception : std::exception

{

virtual const char* what() const throw () { return "error"; }

};

class inc

{

public:

inc(int m) : m_max(m), m_count(0) {}

int go()

{

if (m_count > m_max)

throw my_exception();

return ++m_count;

}

private:

int m_count, m_max;

};

int main(int argc,char** argv)

{

int count = 1000000000;

try

{

inc f(count);

int x;

while (1)

x = f.go();

}

catch (std::exception& e)

{

printf("%s\n", e.what());

return 1;

}

Before we cut to what you're waiting for, the benchmark, let's look at a couple things here.

First, the exception version is more self-documenting. When the error occurs, it's clear what the error is. It also catches more errors than just the one I'm aware of. Any other standard exceptions will be caught and the program will then exit gracefully with a possibly helpful error message. To accomplish this with the return code version, we would have to setup a error code system with associated enums, functions, etc. With this system, we just hook into the already created std::exception class.

Second, the code for the exception version is more elegant. Take the signture for the "go" function for example. In the return value version, I'm already using the return value for an error code, so I have to return the value by parameter reference, which isn't the natural way you'd want to do this. The exception version does the natural thing and just returns the value.

Now, the benchmark. I didn't do lots of testing with lots of compilers and so forth. Feel free to do so and torture me mercilessly, but I have my doubts that you'll get widely different results. On my machine here at work, an i7, the above code runs in about 4.8 seconds when using return values and 3.6 seconds when using exceptions, a 33% improvement.

Passwords Must Die

2010-04-29T15:00:00.001-07:00

Let me be clear and concise to begin. Passwords have failed as a mechanism to secure information technology systems and data. This is obvious to anyone who doesn't work in IT. Users, who are our customers, hate passwords. They hate having to remember them. They hate how complicated we make them be. They hate having to change them every third Tuesday before the new moon. They hate them.

Users who hate features, work around features, passwords included. No amount of poking, prodding, screaming or punishment will make users use features they don't like. Especially when they don't actually work for you. We see this in the different password policies at most businesses versus those for most web sites. Web sites typically want to keep their users around, so they don't require particularly onerous password policies. Corporate password policies on the other hand border on the insane. I'll admit that they're likely necessary to secure the systems, but if we really have to go to that much trouble, then is it working well?

No. Again, passwords have failed as a mechanism for securing systems and data.

We need something new. We need to throw away the old way of doing things do something that makes sense. Something that gives users a simple and easy way to secure their systems. We need to step back and ask ourselves, "Is there a better way to secure systems and data from unauthorized access than passwords?"

I think there is better way to do this. It's already been in existence since before computers were even invented. They're called keys. Physical keys that you keep in your pocket. They work amazingly well for almost all the purposes that we need passwords for.

Why do they work so well? Part of their advantage is that they are physical. They can't be passed around easily. They're easy to secure. Just keep them in your pocket. If you want to loan them temporarily to someone else, you can do that, and then get them back. Of course, someone can copy those keys while they have them, but like anything, you have to be able to trust people not to do that. If you don't trust them, then don't give them your keys.

Of course, if you lose them, you're in trouble. So people do their best not to lose them. And if they lose them, then they make sure to secure the items that the keys gave access to. They change the locks on their house. They change the locks on their cars. It's a pain, but they're willing to do it because they don't have to do it very often, and they realize that it's their own fault for losing the keys in the first place.

None of this is true for passwords. Passwords are hard to secure if you want to loan them to someone. They're hard to secure. Good passwords have to be kept in the human brain. The problem is that good passwords are complex and thus not easy for the human brain to remember without training. So people write them down, which makes them hard to secure.

In our world of information technology, the user's opinion is given a lot of weight, as it should. And they hate passwords. Why are we ignoring our users? Microsoft Office users said they hated Clippy, so Microsoft got rid of Clippy. Users are telling us that they hate passwords, so why aren't we getting rid of passwords? Why are we trying so hard to "improve" them?

We need to get rid of passwords and replace them with keys, physical keys. Many of us already carry around a very appropriate key that could do this jobs. They even called keys. They're called USB keys. Almost every computer around has USB ports. Why can't we use USB keys as the physical keys for our computers?

This can work. What it requires is the information technology community, from operating system vendors and web browser developers to USB key hardware makers to create what would be a very simple standard for storing the keys on the USB drive and communicating them back to the program or system that's asking for them. In fact, we can continue to use passwords. The passwords just need to be stored on the USB key.

The problem is ubiquity. Keyring software exists. But it's hard to use. It's mainly designed for the paranoid tin foil hat types. The type that think that at any time the NSA and FBI are going to knock down their door. It's complicated and hard to use. It requires passwords on top of passwords, which defeats the whole purpose. I can't use it to log in to Windows. It doesn't automatically fill in forms on websites.

This software really isn't something that you want as a tack on program. It needs to be integrated into the OS. This idea of talking to a device to get authentication keys needs to permeate everything. New systems need to be developed with this in mind. They should require a system like this. They shouldn't even have a password field. They should ask the device directly.

We as information technologists need to take a more pragmatic approach to security. If our users aren't part of the solution, then they're part of the problem. If they have to leave the back door open so they can do their job, then we have failed to secure systems, because the job is the most important thing. When security systems fail because of password problems, it may not be our fault, but it is our problem.

My Confession

2007-06-22T16:03:00.000-07:00

I have a confession to make. Forgive me. Wait, don't forgive me. I'm completely unapologetic. I am a programmer, and I don't write comments. I just needed to get that off my chest.

I don't believe in comments.

I have been writing a lot of fairly complex code lately. And all the while, the voices of dead Computer Science professors have been speaking to me. They repeat the mantra of good code commenting. I feel guilt, like when I go to church. Or when I don't make my bed in the morning. Of course, not one of them is able to give me any good suggestion of what a good set of comments is. They just tell me what isn't a good comment. So does that mean that anything else is a good comment? Like lots of swear words in the code. That's probably more useful than real comments, because they make me laugh and keep me from falling asleep at the keyboard.

Good comments, I'm told, are not just a rehash of what is already in the code. Well, if it isn't already in the code, then it isn't much use to the program is it? I don't believe in comments. I think they are mostly a waste of time. Maybe not for you, but for me they just make my life difficult. I have to make a context switch to English in order to write them. That takes time and just serves to confuse me.

Whenever I write English, I take the audience of my writing into account. Who is the audience for my comments? Some moron with a basic C++ book on his desk? Or the great man himself, Bjarne Stroustrup? Bjarne is pretty smart and will probably be able to figure out my code just fine without me, or my comments. Because he speaks C++. I speak C++ too, so that's how I like to communicate with computers and other people who speak the same language.

So I don't write comments. I'm one of those people who likes to use good variable names, good function names, and good file names. When I look at others' comments, I don't usually trust them, because they often don't make any sense. Or they are just plain wrong. That's just awesome. Like the time I first starting programming and I spent two days wondering why the second member of a pair of ints (pair) was always zero, even though the comments said it should contain some valuable piece of information. Actually, it was the first member, not the second one, which I finally figured out by actually looking at the damn code. Wonder of wonders, the code actually told me what the code did. Amazing.

I think that instead of comments we should put quotes of great authors at the top of all our code. That way, when people read our code, they will think that our code is profound, because we quote the greats of our time like, Dostoevsky, Helen Keller, or Dave Barry. And the best thing would be to just randomly pick those quotes so that when people try to make some connection between the code and the quote, they'll spend lots of time trying to figure out. Then they'll feel stupid, but won't want to admit it and we can fun of them when they can't explain the connection. And we won't have had to be smart at all, because all those people that we referred to are smart.

Have I even written comments? Of course, I slap all my comments in the headers, when I don't feel like writing documentation. Or when the function name is getting too long. Or when some fellow programmer makes me feel guilty for not following the religion of comments. What is the point of writing commments if the function name tells the whole story? Take vector for example, the size() function returns, guess what, the size of the vector. I know what you're thinking, that is completely non-intuitive. It's got to be commented. Look, if the function name can't tell you what the function does, then maybe you should change the function name. And if your function name gets too long, then maybe your function is doing too much.

Good, maintainable programs are easy to understand not because they have lots of comments explaining their complex structure, but because they are straightforward, not complex. Complex is a synonym for spaghetti. Code should be more like ravioli. Good ravioli, not that crap they give you in restaurants. Chef Boyardee is delicious. The mental model of a good program is easily understood by normal humans. And if that mental model is nice and straightforward, then the functions that act on it should also be straightforward. A function called get_ the_ thing_ and_ reroute_ the_ other_ thing_ with_ the_ thing_ you_ have_ from_ before() doesn't need to commented, it needs to be thrown out. The class it lives in probably does too. It's doing too much.

I'm religious, but not about programming. In religion, there are things that you believe just because they seem right. Code comments may feel right, but they just don't prove their value outside of giving people a good feeling. I'm going to love my enemy for no good reason, because that sounds like a good idea. But I'm not going to type any more comments until I have a mathematical proof that it's good for me, like ravioli.

C++ is not dead.

2006-01-10T11:10:00.000-08:00

There has been some recent speculation about the death of C++, since the C/C++ Users Journal has ceased publication. So is C++ dying?

Maybe.

C++ as the one and only language for application development died a long time ago. Java has taken over what once was COBOL's place as the language of business logic. If you're writing straightforward code that is going to be used by the five other people in your company, then Java is for you. It holds your hand and doesn't let you make many mistakes. It has lots of libraries for those who have no desire or brains enough to write their own. Not that I think Java programmers are stupid, some of them aren't. I think that Java, like many other scripting languages, allows you to be stupid, and still write code that works ... mostly.

But yes, Java should take over the role of "Business Language for Dummies". C++ was never good at that. But since there wasn't anything else around at the time, that didn't stop people from trying. Which is why we have so much C++ code that sucks. A lot of stupid people tried to use a language that requires some brains. C++ isn't for dummies. It's for people willing to take the time to learn the language. And as I've said before, that requires understanding of C++, not just C. So C++ gets a bad rap.

Some people say that C++ is dying because C does everything that systems developers need to do. Well, so does assembly, but they only use that when they have to. C++ adds a lot of features that most systems developers don't even know about, let alone know how to use. Real programmers only need while loops, but that doesn't mean that for loops aren't useful. How about template metaprogramming? How about virtual functions instead of function pointers? How about references instead of pointers? Can you create a new numeric type in C and have it act natively? And get the compiler to optimize and inline all of that code? Can you write recursive algorithms that can get inlined completely?

All the excuses claimed by C wielding systems developers come down to two basic problems:

1. There isn't a C++ compiler for my platform.

2. C++ is hard to learn. All them fancy templates and classes are just too complicated.

Ok, some systems developers may have an excuse on number one, but since g++, one of the most standards-conforming compilers out there is available on like 20 platforms, chances are your processor is supported. So it's less of an excuse than you think. It's probably going to optimize your code a whole hell of a lot better than the craptacular C compiler that came from the vendor that barely compiles to correct assembly half the time. I love to hear the words, "But C is faster than C++.", coming from someone who is knowingly using a substandard C compiler. If your compiler sucks, then does it really
matter what language you use?

Number two is just ridiculous. Anyone who has learned about C, pointers and recursion, should be able to learn the features of C++ with some effort. Will it take a little work? Yes. But learning C took some work too, didn't it? C++ is the next step. Really learning C++ is going to take as much effort as learning C did, even with your C knowledge.

So C++ is dying because Java is better for writing code to organize your CD collection on the one end and C is better from writing systems code on the other end. In the middle we have real application development, which C++, turns out, is pretty good at. How do I know this? Because as a manager from Microsoft once told me, 90% of code at Microsoft is written in C++.

No, actually, I don’t think that C++ is dying. Spare me.

CSS is Crap

2005-09-07T21:10:00.000-07:00

Ok, so I think it's fairly obvious that I am big fat C++ bigot. That's probably because I've programmed in C++ more than any other language. So I'm a little biased. That's okay because C++ actually is better in many cases.

But because I'm the resident "computer guy" for various groups, like my family and friends, I regularly step out of the C++ realm and try other things. Take for example, web pages. I've done my share of JavaScript and PHP generated pages. But most of that was years ago before what I have been told is now the golden age of web development. Life was hard then, but back then we only had IE and Netscape. But a new day has dawned. A great leap has been made on the Interweb. With technologies like XML, CSS, XHTML, PHP, and the vast Java pile of beans, cheese, and who knows what, developing web pages is supposed to just be a piece of cake.

Recently, I tried to actually create one of these fantastic web pages using one or more of these technologies. After about a week of just trying to format a page without using those evil nested tables and using the magic of CSS, I started to cry for my mommy, the only person who might care about my personal hell.

Why in heaven's name does this have to be so damn complicated? I've learned numerous languages and file formats over the years, so I think I have an idea of how to go about reading a language spec and understanding what it does, and then how to use it effectively. Has anyone actually tried to use the CSS spec or any of the docs out there to get anything done? No wonder there isn't a browser out there that actually implements it properly. It is so generic as to be completely useless.

I really feel for the children of those wrote that stinking pile of MIT-grad generated uselessness. I'll bet they tell their kids when they ask for math help, "Okay, I'm not going to actually tell you how to do that math problem, I'm just going to give you these advanced level calculus books and let you figure it out yourself. It's all very clear once you understand the analysis of infinite-dimensional ring theory."

Yes, I understand that there should be many ways to skin a cat, but CSS gives you about seven hundred. And that is just the number of ways to tell it how big to make your fonts. Not that any of them actually allow you to do that. They all just give the browser a suggestion. And since we really do need 17 browsers out there (for competition right?), there are 17 different ways to interpret what I put in my CSS. In other words, my page looks like crap on anything but Opera version 6.6.6.

I missed the dot com boom. I was working. Is this the nonsense that they spewed out with all those billions? Look folks, this is what you get when let philosophy and foreign film majors get into engineering. Heaven forbid that someone disagree with someone else and the person with the better solution actually win the argument. That would be solutionism. Almost as bad as racism, ideaism, or using your brain. They put everyone's solution in there. Just in case people want to use CSS to teach their dog to hate kids with green eyes.

All those spare web developers are "rethinking their careers" because they sucked at them. No wonder they didn't get a lot of fulfillment from their jobs as web lackeys. If I sucked at something, I don't think I would like doing it either.

Some people just need to say, "That sucks. Really it does, and here is why." They want to think that everything has a meaning and value. News flash, some things just suck. They're crap. They should be thrown out. Figure out the right way to do the job, and leave everything else behind. Like 90% of CSS.

And your little dog too.

Java is the worst of both worlds

2005-06-26T21:43:00.000-07:00

Unless you were sleeping on the first day of your programming languages class, then you know that there are two kinds of programming languages, compiled and interpreted. Compiled languages are those that are routinely converted to some other language (usually assembly/machine code) for execution. Interpreted languages aren’t. Another program interprets the code on the fly and makes the program work. And no, pseudo code is not a programming language, it’s a human language. Don’t even dare confuse the two. See the entry titled “Humans are not Computers” if you disagree with me. You will realize the error of your ways.

Both types of languages have their advantages. Compiled languages tend to be able to squeeze the most out of their host platform because there is no interpreter to slow things down. However, much of the “information” about what the program is supposed to do is lost since we’re converting from one language to another. You always lose some insight when you convert from one language to another.

Interpreted languages on the other hand tend to be more flexible and easy to use. Because interpreted languages don’t have to worry about what the code actually does until it has to do it, it can allow a lot of ambiguity. The interpreter can figure out a lot more things by itself if it has the data that needs to be processed at hand when it interprets the code. A lot of flexibility comes from the ability of the interpreter to interpret any code given it, even if that code didn’t exist until runtime. The PERL eval() function takes PERL code as a string and can interpret it on the fly. Also, interpreted languages are executed on the fly, the interpreter has more information about what the programmer meant to do, so it can optimize many things that a compiled language can’t. This capability hasn’t been exploited as much as it might be. The Java folks have done some work with just-in-time compilation. I suspect this is where they get ability to hit benchmarks as well or better than C++ code.

Ah, Java; the savior of all of use who are trying to write software for everything from mainframes to calculators to cheese slicers and unmanned exercise machines. Java is the worst of both of these types of languages. It isn’t an interpreted language, it’s compiled. But it isn’t compiled to machine code to take advantage of native code, it’s compiled to Java byte code, which is then interpreted. “But byte code interpreters are really fast!” they say. I don’t care how fast they are, unless they’re so fast they don’t actually run, I don’t much care. If they take time, they take time. You just can’t get around that.

Now I have nothing against interpreted languages. PERL is one of my favorite tools. But look, Java just doesn’t give you any of the flexibility of an interpreted language. Once you’ve compiled to byte code, you’ve gotten rid of all that semantic goodness that is found in the original program. So you’re interpreting without any of the benefits of interpreting, and compiling without any of the benefits of compiling.

I hear folks saying, “what about the platform independence? Isn’t that a great thing?” Yes, yes it is. But there a lot better ways to solve that problem. On the compiled end of things, you can use platform specific code wrapped by platform independent APIs like STL, Boost, Qt and wxWindows. Or just write platform independent code with conditional compilation (#ifdef’s for the buzzword impaired).

On the interpreted end, things are even easier. Just write platform specific interpreters. This is probably the way to go when performance isn’t an issue. And even if it is, then JIT techniques might help your cause.

Java just isn’t the answer.

Writing C in C++

2005-05-17T20:40:00.000-07:00

I’m sure that you’ve heard of the term C/C++ as if somehow they are so interrelated that they can combined into a single term. C and C++ are about as close as a lilac bush and George Bush. Sure they share some of the same terminology but you wouldn’t want to meet one of them in a dark alley at night.

C is good for one thing, being one step above assembly code. Anyone who tells you otherwise is lying to themselves. Everything about C is about making things you do in assembly easier. The perfect example is the switch statement. There is no rational reason to force the programmer to put break statements at the end of every case. It doesn’t make any sense unless you know that the switch statement is likely to be implemented using jump opcodes. Also, who in their right mind uses up an entire operator to implement bit shifting?! Unless you’re some sort of divide-with-shifts freak, then you’re just not going to use them very often.

And why do they call them function pointers? There’s no reason that the user should have any idea how functions are implemented, let alone knowing that they can be pointed to. The syntax for function pointers is bad enough.

C is a great language for compiler writers. For every construct in C, there is a fairly straightforward translation to assembly. This is of course the main issue behind all the complaints about C. It wasn’t designed to be a language, it was designed to easily translated. So it shouldn’t come as much of a surprise to learn that the guys who came up with templates never wrote a compiler. Sure, they took a class on compilers, but that doesn’t mean that learned to love their local compiler guy. Really, who complains the loudest about how hard it is to implement templates in C++? Compiler writers. More specifically, Microsoft’s compiler writers who proved that templates are hard by producing Visual C++ 6.0 where using templates is like trying to teach your dog new tricks only to find out that it is an armadillo. Sure an armadillo looks like a dog, but partial function template specialization looks likes it might work. But it just doesn’t. Whether you like it or not.

C++ is a boon to the programming world. It took C places way beyond where the compiler writers wanted to go, which is a good thing. When people start complaining about something, it’s usually a good sign that something is getting done. C++ doesn’t make any effort to be nice to compiler writers. This made getting Visual C++ done take way longer than Microsoft Bob, but it sure makes for a flexible language. Compiler writers don’t like that word, flexible. It means that they have to work. They have to work to take all the flexibility into account. But I don’t care, I’m not a compiler writer.

Someone invented C so they wouldn’t have to code in assembly, and someone invented C++ so they wouldn’t have to code in C. So if you’re a C programmer writing C in C++, then you’re basically installing an automatic garage door opener and then opening the door by driving into it. You’re not using the features of the language that you claim to embrace. Here are some of the features that C in C++ programmers tend to continue to use even thought C++ has surmounted their craziness.

First, function pointers. Virtual functions were invented so you freaks using function pointers could stop the madness. If you’re using a function pointer as a callback, then use virtual functions. They’re easier to use, more flexible and the syntax doesn’t look like monkey spew. There are numerous other reasons to use virtual functions beyond callbacks, so for those of you who think that function pointers are God’s gift, they’re more like Satan’s spawn.

Second, typedef’ed structs. Newsflash, typedef’ed structs are no longer necessary. You wouldn’t be using them in C++ if you knew why you had to use them in the first place. A struct is now an actual type. That was the whole point of C++. Making classes and structs act like native types. Along with this is the flawed belief in the difference between structs and classes. Repeat this mantra every morning in the shower if you still think that stucts and classes have some fundamental difference, “A class is just a struct where the default is private versus public.” There isn’t any other difference. No way, no how. Most compilers don’t even remember which is which.

Third, void pointers. They were a bad idea in C, but perhaps a necessary evil. But in C++, they’re just not at all necessary. Void pointers are not the tool of the super-hacker, they are the last resort of the design-impaired. Just because you don’t know of any other way to do what you’re trying to do with void pointers, doesn’t mean that using them is OK. It’s not OK. C++ provides so many ways to not use void pointers, that using them is a sign of your C++ ignorance. Go back ten spaces and read your beginning C++ book again.

Finally, using pointers to pass things by reference. I don’t care that you’re used to doing it this way. We just don’t pass things by pointer anymore. We pass them by reference. References are an improvement over pointers. Get over that they can’t be NULL. This is a good thing. 90% of crashes in C programs are NULL pointer dereferences. So it makes sense (yes, it actually does) that you should want to minimize the use of pointers. They don’t make you cool or more of a super-hacker because you use pointers for everything. They just make you reckless. References are safer than pointers. You don’t drive around your car without a hood just so you can have easy access to the innards when you need them. You open it as needed. Adding pointers is just like putting hood pins on a Pinto. Same goes for pointers. Use them as needed.

So what is the message here? If you’re using extern “C” at the beginning of your C++ code and telling your boss that you’re writing in C++, for all the rest of us who didn’t sleep through the nineties and have to read your over-syntaxed code, please go read a beginning C++ book and join the 21st century.

Humans are not Computers

2005-03-28T21:52:00.000-08:00

I read an article today about a program that is supposed to turn English language into code. Many folks thought that this was a great thing in that people would become more precise in their use of English as a way of describing what a program does. This is wrong and a complete misunderstanding of language, both computer and human.

Here's a news flash, just because PHP and Swahili are both called "languages" doesn't mean that they really have much to do with each other. The reason that they are both called languages is because they both express something to something else. They're completely different beasts. Computer languages are for talking to computers. Human languages are for talking to humans.

Computer programmers are fairly antisocial, and so don't have a lot of experience actually talking to real people. They spend a lot of time expressing themselves to the little boxes on their desks. These boxes are not human beings. Humans don't work like computers. They don't do what you tell them to do most of the time. They won't always like you. They might just up and leave if you don't treat them right. And the biggest thing of all, they have brains. No really, your computer doesn't have a brain and that annoying marketing guy does.

This means that no matter how many times you snicker or roll your eyes behind their back about their lack of knowledge of Excel macro programming, humans are smarter than computers. They might actually be smarter than you in some area other than computers. For example, they might know more about something important, like how to make money, or why you should take a shower, or how to talk to girls.

Back to the main point, which is that there is and should be a difference between human and computer languages. Forcing people to be more precise in their use of human languages only forces them to think like a computer. Why should they think like a computer? They're not computers, they're people, and people are smarter than computers.

Your job as a computer programmer is as a translator between human language and computer language. Anyone familiar with translation of human languages knows that it takes some brains to do it. You have to be familiar with the idea of context, idioms, etc. of both languages in order to translate well. To be a very good translator, you also have to be familiar with the cultures that you're dealing with. Something a computer will likely never do.

The same goes for computer programming. Computer programming isn't simply about translating word for word the human spec to code. You must interpret the human language by using your brain, and convert it to computer language. A spec is written by a human is meant to be read and understood by a human. The spec writer isn't there to do your job. That *is* your job, translation. Programmers who don't understand this concept should be beaten and have the word "Spec Freak" tattooed to their forehead. If you are a "Spec Freak" and no one knows about it yet, then take this opportunity to stop. People might actually start to like you. They might not run away as you approach the water cooler. And if you combine it with regular showering, then you might even be able to start talking to girls.

I'm going to tell you a story about UML. If you're not familiar with it, it is a way of describing the design of code using pretty pictures so that PHB's can understand it. It actually turns out that when used properly, it can be an effective tool to communicate to other humans about the design of your code. It's a human language.

The problem came about when some lazy "Spec Freak" was annoyed that after he spent all this time creating a pretty UML picture that he had to then translate the picture into code. He thought, "I should write a program to do this for me." He thus sealed his status as a "Spec-Freak". He then found that there were ambiguities in his UML that couldn't be translated into code. He found that they were things in his code that he couldn't translate into UML. So he "fixed" UML to be more precise, and more useless. UML is a human language. Human language has a lot of ambiguity, because humans are smarter than computers, and can deal with it. Adding precision and getting rid of ambiguity in human language turns it into computer language and thus gets rid of all its usefulness as a way to communicate with other humans.

Humans aren’t computers and computers aren’t human. Humans communicate well with humans. Computers communicate well with computers. Programmers are needed to translate between the two. No computer will even be able to do it. At least not until strong AI gets invented.

Optimization

2005-02-22T10:40:00.000-08:00

I don't think that I would make a good game programmer.

A former coworker's life dream was to become a game programmer. Well it came true and I found myself wondering why someone actually hired him. Look, I've seen his code and have to work with it every day. Or more accurately, I play chicken with his code every day. I poke at it in some soft spot hoping that it will give way, only to find myself being thrown off in some other unforeseen direction. His code can't be modified without breaking at least 2 other things. It's like wack-a-mole. You pound down one of the little buggers only to have two more pop up.

So I assume that they took a look at his code at some point during the interview process. They must have seen the consistent use of sweet optimizations, like bit-shifting instead of dividing, reuse of variables because they cost so much you don't want to waste them, and using C arrays instead of std::vector because, well, C arrays just gotta be faster, because they're C. Being game programmers themselves, they probably saw the incredible value of all this. They also saw that he refused to cave into the C++ madness and still coded in C, using a C++ compiler. Because, it's just faster you know. All those objects are just too useful to be efficient.

In game programming, it is apparently a good idea to make calls to your database in an inner loop. My coworker did this all the time so it must be a good idea. I think I’m just uninformed, because last time I checked, database calls are pretty slow. I guess if you wrap them in a function that doesn’t *look* like a database call, then it’s okay. Or maybe simply putting them in an inner loop magically makes them faster. I wish I knew more about game programming. Maybe I can buy a "Game Programming for Morons" book at the mall and start working on becoming the next John Carmack.

Okay, so seriously, this coworker of mine actually got in an argument with me over the value of using bit shifting over a simple divide for integer division. Yes, I know everyone is taught that bit-shifting is really fast, but while I understand that no official announcement was made, compilers are now pretty good. They’ll recognize your divide by a power-of-two and convert it to a shift while you retain the readability of a divide. They’ll even make sure that it works for signed and unsigned numbers correctly. I even proved this to my coworker by showing him that the disassembly from even the MSVC++ non-optimizing compiler optimized this correctly. His response was something along the lines of, "oh, well, right, yea, but, well, what about pipelining?" Huh? Pulling out the technology of the week doesn't help you win the argument. I wish I mean enough to explain to him how stupid he sounds.

This is of course the same guy who doesn’t see a problem with casting to integer what should be simply left as floating point, because you know, integers are faster. We’ll just ignore the fact that casting from floating point to integer and back takes *forever*. And in the same code as he does his cast to integer, shift to divide (because it’s faster), and cast back to floating point, he makes one of the biggest game programmer gaffes, he writes:

pow(x, 2.0);

I mean, come on! This is why this blog is called Mad Software. This stuff just makes me want to scream. This isn’t just a game programmer thing, this is something that no one does. They just don’t. It’s far faster and just as readable to write:

x*x;

Now, this whole rant brings us to one of the first lessons of optimizing your code. Don’t optimize prematurely. All the above nasty is a perfect example of this: lots of optimizations at code level, without ever thinking of the global optimization problem.

You probably didn’t believe me when I said this guy was putting database calls into tight inner loops. Well, you’d be wrong. It was cleverly hidden in a conversion function, but it was a database call all the same. He never bothered to optimize it, or even did he recognize the fact that it was slowing the whole program down. That’s because he never did what a good programmer should do, profile his code. That is to find out what parts of the code actually are taking all the time. Guess what, it probably isn’t that really slow divide by two. It’s the database call hidden by a few layers of indirection that you’ll never see because you didn’t write the database code and didn’t realize that the guy that did is an idiot.

I assume that my coworker’s new employers saw these kind of things in his code because game programmers are the cream of the crop and don’t miss anything.

I just wouldn’t make a good game programmer, my code doesn’t suck enough.

Getting into Software

2005-02-21T19:00:00.000-08:00

Ok, here at work, we've been looking for new developers. And the range of resumes reminded me of one of my favorite mad rants. That rant is all about these people who "get into" programming, as if it's something you can figure out in a weekend and do a great job at. They wear the badge of "self-taught" like it's a *good* thing. Why is it that people think that one of the most intellectuallydemanding professions in modern times is something that one can simply read a couple books about and be an expert? The software development world is full of people who have degrees in things like philosophy, civil engineering, and english who at some point realized that they weren't making any money in their chosen profession and so "got into" computers. Guess what folks, my two year old can "get into" computers. But that usually ends up with me extracting pieces of fruit snack from drive bays.

Just because they've read a few books down at Barnes and Noble doesn't mean that they're going to be a pro software developer, or the next Linus Torvalds. Here's what they say in defense of their untenable position, "There are people without degrees who have been very successful in computers without training." True, but those folks are either very smart, or very lucky. The reality is that most people aren't that smart, and by definition, most people aren't lucky. They just need to stop thinking that magic is going to happen, and they're going to become Master Programmers by reading "Get Sweet Game Programming Skills in 21 Days". Just because one can follow some instructions on how to write a program to compute the area of a circle, doesn't mean that you're going to writing a word processor anytime now.

Programming is hard. Programming requires some training. Programming requires someone who can think through difficult problems and explain them to a computer.

Another thing that people just "get into" is management. This is another profession that while may seem simple, actually requires some training to do well. Why do you think people get their MBAs? For kicks? Wrong. Amazingly, an MBA is actually taught things in those classes he's taking. He's learning lessons learned by others in the real world. He's learned theory about economics, so he can decide how much to pay their employees without bankrupting the organization. Why are there so many incompetent managers out there? Same reason that there are so many incompetent programmers out there. The low levels of the profession are easy enough to fake that almost any nitwit can get by without actually knowing anything about the profession itself.

Back to programmers. This is all about reinventing the wheel. When a scientist begins his work, he doesn't spend ten minutes at www.physics-for-dummies.com reading a tutorial and then proceed to make some "sweet experiments". They spend years in training, and then after all that, they still spend much of their time learning about their particular area so that they can build on the work that has been done before them. Many hours have been used to study how best to write software, so that it is fast, easy to use, and efficient. These aren't just some random useless ideas like those thrown around in discussions in english literature classes. They're important useful stuff that someone isn't going to understand unless they spend some time learning them.

You're not going to figure out the theory of special relativity by thinking about light and time. Einstein could do that. You can't.