Cold Electrons

I’ve been writing a program to work on textual data piped through it from tshark.

Most of my struggling has been fiddling with regex libraries, but now that I have it working, I have noticed two things:

When piping a test file through it, CPU usage stays pegged at 100%, even after the file is done and the program is waiting on data that will never come.

When used as part of a pipe from tshark, CPU usage remains low.

So, I probably need to add something in to peek into std::cin to see if data is waiting and then sleep the thread, or check on the status flags of std::cin.

I am using gcc 4.6 or gcc 4.7, as well as trying out stuff in cling.

Today I have learned the following:

1. The standard library regex with gcc 4.7 is broken.
2. Poco 1.3.6 (with debian wheezy) RegularExpression library is broken.
3. boost 1.49 regex (with debian wheezy) works.

First I tried the standard library, because it’s, y’know…standard.
I was playing around with it in cling, but it absolutely wasn’t working right.
I couldn’t even get a string to match itself; it’s about that point I went looking for some changelog or news and confirmed that it wasn’t going to work.

Next, since I was already using the wonderful POCO project library to take care of the boilerplate of making a CLI program with the Poco::Util::Application class, I figured “Hey, it’s got a regex class! I use it, too!”. Wrong. The Poco::RegularExpression search and extract methods would return a non-zero count of matches, but attempting to extract the matched string would always be empty; this was true in either cling or my compiled code.

So I knuckled down and prepared to face the slightly uglier syntax of using boost. It’s not that bad, but I can’t use any part of boost in cling, due to boost using inline assembly.

Premature optimization is the root of all evil.

I’ve heard this before as have other people. As an overly-cautious person, I tend to think out my approach to things, and sometimes I have to forcibly tell myself to stop.

Case in point: Using std::vector in C++.

First some things I’ve in various documentation, forums, websites, etc:

• Vectors are dynamically-allocated, variable-length arrays.
• Vectors try to keep a certain amount of capacity to allow themselves to grow without reallocation.
• Capacity does not shrink, even if you empty the vector.

For those who come from a plain-C background, this might make you think that a C++ std::vector is very inefficient. Oh my, I’m wasting memory! Oh dear, I’m getting unnecessary memory allocations! Oh god, I can’t reclaim the memory!

So some people ask, on StackOverflow and the like, about making a fixed-size container, saying that vectors are too ineffecient.

But are they? Has anyone ever actually checked to see how the std::vector behaves in use?

So…the reason I bring this up is that until a few minutes ago, I was one of those people, trying to optimize the hell out of something, without knowing if it really needed it. I had a case of needing a container, but only needing to either a single element, or a large-ish (4096) number of them. Messing around with other things was proving messy – I am simply too lazy to muck with pointers and multiple types if I don’t have to.

So I wrote a quick test to challenge my assumptions. Here’s the output:

$ ./a.out 
Testing vector size and capacity.

vector chunk: size 0, capacity 0
Let's grow some...

size	0	 capacity	0
size	256	 capacity	256
size	512	 capacity	512
size	768	 capacity	1024
size	1024	 capacity	1024
size	1280	 capacity	2048
size	1536	 capacity	2048
size	1792	 capacity	2048
size	2048	 capacity	2048
size	2304	 capacity	4096
size	2560	 capacity	4096
size	2816	 capacity	4096
size	3072	 capacity	4096
size	3328	 capacity	4096
size	3584	 capacity	4096
size	3840	 capacity	4096
size	4096	 capacity	4096
chunk.resize(0)...
vector chunk: size 0, capacity 4096
Now let's try a little swap()
trick to shrink it:  vector().swap(chunk);
vector chunk: size 0, capacity 0
Now let's try that again,
using chunk.shrink_to_fit()...
vector chunk: size 0, capacity 0

Staring aghast, I shamefacedly realized that I had been trying to be too clever for my own good, and quietly muttered thanks to Stroustrup and the countless other people that provided me with such wonderfully powerful tools to use, and belatedly reminded myself not to assume that I could so easily out-think so many professional software engineers.

Vectors are good. Use them.

And not in a fun way.

I have, of late, been unable to function in levels of higher-order intelligence. For example: A task, done innumerable times prior, with a simple order of steps to proceed – and I am completely unable to do it. I try to think of what I should do next, and I can’t formulate it. I try to visualize what it is I’m trying to accomplish, and nothing happens.

All math skills, verbal skills, motor functions, and sensory inputs are unimpaired…but foresight and planning are right out the window.

I used to take pride in the fact that my mind was disciplined enough that the old word-association gimmick used by dumbass psychologists produced no involutary reaction – especially if I didn’t want it to. You could say to me, “Don’t think about elephants”, and I wouldn’t. If you asked me what I thought about X, and if I answered, “I don’t.”, I meant it quite literally.

Basically, I am currently in a good day/bad day inversion: Instead of most days being good with a few days here and there when you’re off your game, I am having most days I am a useless lump of meat, with only a few here and there where I can actually pull my head out of my ass and get stuff done.

I’m not foggy or drugged out or even that tired. I feel (for lack of a better term) normal, except for the fact that I simply seem unable to focus on performing work.

What’s worse – if it’s possible to be worse than going balmy – it that I’m working with an old friend, and he knows that I’m not a stupid, incapable schmuck, and he’s getting pissed when it seems that I’m dumber than a dingleberry.

So, I ask the question, with a hanging dread and real fear: What the hell is wrong with me?

Edit:  What was wrong with me?  It seems the major contributing factor was drinking Neuro Sonic.  I’ve stopped drinking that stuff, and my focus and ability is back.

As I’ve recently been working on a game (who isn’t, these days), my mind has often been turning to that ‘other’ operating system – you know, the corporate one with a major market share? – and how to build my code there.

So I tried that.

Downloaded MSVCwhatever/mingwhatever/cygwhatever and tried them all.

I think I threw up in my mouth a little.

Yeah, one can build code in Windows. However, I can’t imagine actually working in that environment. The lack of a good terminal and shell…just…[shudder].

But I digress.

Back in my comfortable hobbit-hole-os, I began to take a harder look at using a cross-compiler to do my dirty work. What I’d really like is something like Buildroot, but using MINGW, and produce apps and installers, not root filesystems.

The closest thing I’ve found is MXE, the Mingw32 Cross Environment.

I’ve already compiled my first cross-built DLL of RakNet, and tried another (OIS of wreckedgames.com), and the major points involved in this process that I’ve learned are:

1. Changing and patching the source code of what you want to build is inevitable. This is due to the fact programmers don’t always make code to encompass all possible environments/platforms.
2. For some libraries that use autotools (OIS), it will be necessary to completely create my own build system, because despite passing environment variables and options to ./configure, OIS still tries to build for my linux host, and not the target ‘i686-pc-mingw32′.
3. Once I start down this path, forever will it rule my destiny – libs built with one compiler are not necessarily binary compatible with libs built with another

MXE eases my pain – but there will still be pain. Why would it be worth it? Because once I build the dependencies of my code, I can work in my favorite environment, and compile my code for the other platform as often as I would for my native host.

However, before I masochistically flagellate myself with barbed wire, there is one other method – virtualization. If I can have a good (preferably remote ssh) shell, then I can push code to and compile upon Windows – but the amount of pain is debatable and difficult to estimate.

I did my reading and found two things to do to get USB 3.0 working on my gaming laptop.

I’d tried poking and prodding at my Debian before…but previously I chalked it up to the my stock kernel not being new enough.

However, I just compiled 3.3.0 (which, on this wonderful machine, takes very little time!) and still no joy from the blue port to my right!…I realized that there may be more to this.

The two things I found:

1. (as root) Add pci=nomsi to the grub default kernel command line
2. (as root) Add xhci_hcd to /etc/initramfs/modules, so it gets loaded before any of the other usb modules

Follow these with (also, as root) update-initramfs -u and update-grub.

…I will also admit that I was shotgunning this problem, and I also upgraded my BIOS to the latest version 203, so I can’t be perfectly certain that didn’t do something.

And so, voila!

$ lspci
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5)
00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5)
00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation HM65 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6 port SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
01:00.0 VGA compatible controller: nVidia Corporation Device 1251 (rev a1)
01:00.1 Audio device: nVidia Corporation GF116 High Definition Audio Controller (rev a1)
03:00.0 Network controller: Atheros Communications Inc. AR9285 Wireless Network Adapter (PCI-Express) (rev 01)
04:00.0 USB controller: Fresco Logic FL1000G USB 3.0 Host Controller (rev 04)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)

As part of my recent gearing up for working on our super secret game project, I’ve been decking out my Vim installation with useful scripts.

During part of this process, I found that the OmniCPPComplete script is almost completely unusable when working with the Boost libraries and STL. OCC uses exuberant-ctags to gather information, but the tag files it generates are…rather large (in the case of Boost), and doing autocompletion with STL requires a set of specially modified headers.

While both those problems are not exactly unsurmountable, the main problem I had with OCC is that it is very stupid. Using it with the libraries I wanted to use produce a lot of tag pollution – autocompletion on anything produced an extremely long list of stuff that was nothing near what I wanted. Also, it was completely static: I didn’t get completion on instances, or even of tags in my current file.

So, I discarded OmniCppComplete, and embarked on a quest to use Clang_Complete.

Clang_Complete is an interesting project. Some professional IDEs use the actual compiler to generate the Abstract Syntax Tree of the code – actual living, breathing information about the code as you type it. However, due to some paranoia in the world of the Gnu, gcc does not export it’s AST. Thus, clang_complete uses…clang! Clang is a compiler based on the LLVM.

Luckily for me, even as I was working on this, Debian brought up-to-date packages of LLVM and Clang to my doorstep

However…

As I am working on a multithreaded project, and I’m endevouring to use as modern and as cross-platform code as possible, I hit a stumbling block when trying to use std::atomic:
error: Non-inline namespace cannot be reopened as inline

Clang doesn’t like the nested-namespaces technique used the gnu standard libc++ to version and control what ABI gets inlined. I wrested around for a while, trying to coax clang into getting past this. Nothing was working, so I started looking at it like a platform problem. One platform being actual compiling, the other being clang_complete.

Since my goal was NOT to compile with clang, but only to make autocompletion work nicely for me – and since this so far seems to be only in the files where I want to use atomic operations, I start putting this in the tops of those files:

 // headers like you'd normally expect 
#include <iostream>
// Now for the conditional stuff
#ifdef __clang__ 
#include <clang_complete_compat/atomic.h> 
using namespace std::atomic; 
#else
#include <atomic> 
#endif

Then, I took the regular <atomic>, and started butchering it into something to forward declare what I wanted. I know I could do better, because I’ve missed some detail that entails using the extra “using namespace”. However, for the moment, it works.

Minecraft.  Oh ye gods, Mine-fucking-CRAFT.

I’m not the only person to have such an attitude: One group of modders called their set of mods “Better Than Wolves Mod Pack”.

…masturbatory.

Occasionally, we linux users are bitten by the bug.  Usually, it is when we are in larval stage, and the better of us gain wisdom and avoid it later in life.  Others (such as myself) are locked in a chaotic attractor, spiraling in a love/hate orbit about this amazing system that fascinates us.

The bug-bite I’m referring to is the obsessive upgrading/updating of one’s computer system.

“Dang, my hardware is not supported right now.”

“Ah, there’s support in the latest kernel.”

“Dang, that broke this other software I use. Let’s see if there’s a fix…”

“Ah, bugger…”

Ah, that ‘ah, bugger’ stage of things, where your system teeters on the knife edge of developement. It’s heady stuff, building up your system.

I have recently done something I’ve never done before: I purchased a new laptop.

Not “new used”, “gently used”, or even “a couple years old but still somewhat OK but now found on ebay at half the price” – NEW, out-of-the-box, register-the-warranty, bleeding-edge hardware, not a single fingerprint on it, NEW.

I rather quite like it.

After limping along the past year on my old (but still fully functional) Thinkpad T60, I purchased a NEW Asus G74Sx.

This thing tears my old laptop a new butthole.

However…I still have nits to pick, because I am picky and particular, and there are, in fact, some nits.

The USB camera in the lid. In linux (Of COURSE I’m putting linux on it!), the camera doesn’t like being to enter a low-power or no-power state, and kicks the ACPI system in the balls anytime ACPI tries to sleep.

The keyboard. It’s a nice size, but the layout could use some work. Quite frankly, it’s not as friendly to my keyboard-oriented usage as my old Thinkpad. I miss my Trackpoint, for starters. The ‘delete’ and ‘insert’ key are combined – I have to press a key-combo to ‘insert’. It’s reeealy weird having the home/end/pgup/pgdn keys aaaaall the way over to upper right above the keypad. The pgup/pgdn keys have an issue with responsiveness, and only seem to register every third press. The keyboard backlight is nice, but IMHO there should be a hardware control for the brightness.

The touchpad. Oh gods, the touchpad. I have loathed touchpads since I first saw one. I have since undergone a crash course in touchpad technology, and while I will admit that there might be some good touchpads somewhere, what currently exists in my laptop could not be counted among them. It’s not a Synaptics – it is a Sentelics Finger Sensing Pad, and I have a secret dream that someday I will lift the lid of my G74Sx, and find the touchpad has left, leaving a note: “I understand. I’m gone.” The touch pad is overly sensitive and poorly positioned – as a Trackpoint lover, I hate having anything under my wrists while I type, especially if it reacts to just the near proximity of my thumbs.

All in all, this is a bitchin’ piece of hardware. It’s built like a carbon-fiber-and-aluminum stealth shithouse. It has a nice, sturdy feel, with very adequate cooling capacity and a rockin’ quad-core i7 with 12G of DDR3 and GeForce GTS 560M. Hell, this thing is nicer than my desktop. Still, I may make a winter project of finding a way to replace the touchpad and/or keyboard with something better.