I’ve been toying with a community site recently. The logical starting point for the code behind it is discussion functionality. The idea is to take some existing code that does “forums” well and use this as the kernel that the community site is built around. Nothing is ever so easy though, the ‘net and developer communities have grown so huge that we’re not talking a needle in a haystack but finding the right needle in a haystack full of needles. There’s a scattering of good needles, but a lot of blunt of downright broken ones.

I’ve locked myself into Django now, but even having already determined the web framework to use (i.e. choosing a smaller haystack) the task isn’t trivial. I think that a general issue here is this exact experience I’m having, people give up eventually, try to spin their own, get big ideas, publish their half baked code, and make it just a little harder for the next person trying to find pre-existing functionality. It doesn’t help that Django’s relatively young, I don’t make life easy for myself sometimes!

Here’s my small attempt to work against the trend. I’ve gathered a list of the Django forum/discussion software I’ve found and associated each project with any further useful information that might help the decision of which to use. I’ve filtered out anything that was clearly broken, but don’t know the level of completeness of everything listed here.

Also, I’m sure there’s other projects out there I’ve missed! I’ll update this page if I find anything new.

I’m not recommending any of these, maybe that’ll come later once I’ve actually decided which one to work with.

Ross’s Django Forum (the django-forum)

“Simple Django Forum Component”

This seems to be the most linked-to Django forum software out there. It seems basic, but I haven’t tested it or seen a demo. I’ve seen various blog comments commenting on either it’s “lack of features” or “ease of installation and use” — how many “features” do people really need?

• Google Code: http://code.google.com/p/django-forum/
• Owner: Ross Poulton (ross.poulton) http://www.rossp.org/ (This blog/owner link is a guess.)
• Initial checkin: 2007-04-30
• Latest update: 2007-11-18 (> 1 month ago)

Jonathan’s Django Forum

Almost got this one mixed up with django-forum above. At least it has a demo site! The demo looks like it has all the basic expected components working.

• Code: http://www.jonathanbuchanan.plus.com/repos/forum/
• Owner: Jonathan Buchanan http://insin.webfactional.com/ (There’s some info about the forum in the weblog.)
• Demo Forum: http://incremental.homedns.org:81/

counterpoint

“forum written with django”

Very little information about this out there, the code is available though. I’ve seen a comment on a blog post from Nov 29th 2007 that says “missing much functionality”, it’s a young project though and that was a month ago.

• Google Code: http://code.google.com/p/counterpoint/
• Owner: Collin Grady (cgrady) http://collingrady.com/ (no info on counterpoint at time of writing)
• Initial checkin: 2007-03-18
• Latest update: 2007-12-18 (11 days ago)

snapboard

“Python Forum/Bulletin-Board for Django”
“SNAPboard: S(imple), N(imble), A(ttractive), P(ython) board”

In a way this one seems to do the best job of selling its self. The Google Code frontpage and wiki (documentation) are good. However activity on the project has dropped off and a new maintainer has come on board recently (December 11th according to a forum post).

• Google Code: http://code.google.com/p/snapboard/
• Discussion: http://groups.google.com/group/snapboard-discuss
• Owner: No obvious blog to associate, started by “Bo Shi” and recently changed “maintainers” to a “JW Mayfield”.
• Initial checkin: 2007-01-26
• Latest update: 2007-12-17 (12 days ago, but first checkin in more than four months)

Django Discussion (django-discussion)

“A generic discussion application for Django”

I’m guessing this is the same author as the second item in this list, pretty unlikely coincidence otherwise? Anyway, the code seems to be different, so a different project?

• Google Code: http://code.google.com/p/django-discussion/
• Owner: Jonathan Buchanan (Same author as the second forum in the list I’m guessing, but the code seems very different.)
• Initial checkin: 2007-03-15
• Latest update: 2007-12-08 (only 9 changes since creation, but recently active at least)

Diamanda “MyghtyBoard”

“Diamanda Wiki and Forum”

Diamanda isn’t only a “forum”, it’s a site-builder toolkit that contains a complete “forum” subcomponent called “MyghtyBoard”. The forum seems to possess all the expected features. Development started quite some time ago and was regular up until early 2007, reached a degree feature completeness maybe? A recent “bug report” is “rewrite code for better look and use” with the response “I will in time :)” (6 days ago).

• Google Code: http://code.google.com/p/diamanda/
• Owner: Piotr Malinski: http://www.rkblog.rk.edu.pl/
• Initial checkin: 2006-08-30
• Latest update: 2007-02-18 (quite some time ago, but it had a lot of activity up to this point)
• Also, SourceForge: http://sourceforge.net/projects/myghtyboard/
• Demo Forum: Maybe this is the code in action: http://www.rkblog.rk.edu.pl/forum/

Sphene Community Tools

“Django Forum Application and Django Wiki Application”

Like Diamanda, this is a toolkit that happens to include a forum. I’m assuming that the forum on their site represents a demo of SCT in action. This project looks like the most active of the lot and seems to be quite well documented.

• Website: http://sct.sphene.net/
• Code: http://sct.sphene.net/wiki/show/Downloads/
• Owner: Herbert Poul (kahless) http://herbert.poul.at/wiki/show/Start/
• Initial checkin: 2007-02-18
• Latest update: 2007-12-23 (6 days ago — fixing typos on the eve of the eve, dedicated!)
• Demo Forum: http://sct.sphene.net/board/show/0/
• Also, SourceForge (but of little use): http://sourceforge.net/projects/sct/

ENDE

That’s the lot for the moment.

The last two days have been somewhat joyous in a less than traditional sense. Two whole days with all computers shut off! OK, so not that much different from our recent holiday without work/computers, but more relaxing.

The reason for the title is twofold, firstly the weather here really is rather shit. It’s England! What should I expect? Christmas day was chilly and wet, at least on boxing day there was a little sunlight. Dreaming of a white Christmas around London? Not much chance these days it would seem. This is my third in the UK, the first was white thanks only to a heavy frost, there was a little snow around the period but it wasn’t so cold that I didn’t spend the day on my bike in Wendover Woods. Last year it was cold at least. This year it isn’t even chilly, there isn’t one 0 or sub-zero day predicted in the entire last week of the month! Today has a predicted minimum of 9 — I can quite comfortably wear just a thin t-shirt under an unbuttoned jacket. Oh well.

The other bad storm is one of 2007’s old favourites, the Zhelatin/Storm/Nuwar “worm”. After somewhat of a lull in seeing emails from this network I suddenly got one on the 23rd, as I mentioned on Monday.

This turned out to be the first of many as the network pumped out a full-scale assault capitalising on the jovial season, both Christmas and New Year. Taking advantage in two ways I think: 1) people probably are sending a lot of stupid email right now so it may be more likely that people follow the evil links, 2) A lot of people, including those in the security industry and the IT-shops responsible for maintaining corporate security, are on holiday so the “good guys” may have a slower response time.

The latter point is worth some thought. I’m sure it has been discussed before: computers don’t have holidays, crims take advantage of holidays, most normal people let their guard down on holidays. Good news for botnet herders. As I mentioned earlier in the week the malware payload wasn’t detected by any of the large-market-share AV engines, the biggest player to detect some of the samples I tried was Kaspersky (finding accurate market-share figures is difficult, suggestions on the net for KAV are between 5 and 1 percent). As has now been clearly established, I’d think, the malware writers test against the biggest AV engines. We can get a good picture of which engines they’re testing with by rounding up as many of these jolly-Storms as possible and scanning them to see which engines, when loaded with a pre-mailout database, detect close to 0% of the samples. The list you’ll find isn’t all that surprising.

It’d be really nice to have a good statistic on the size of the botnet on December 20th versus the size on January 7th. But all botnet size estimates are generally a product of bad guesstimation, we can’t expect anyone to know the numbers except the ones in control.

I’m becoming more pessimistic about the situation as time goes on. The concept of a “virus filter” product seems to have been proven fatally flawed. Whether detection takes place via signatures or “heuristics” (in my opinion this is little more than complicated signatures) the approach is reactive. Either to specific malware or to specific exploits, the latter gets a lot of press as “generic” detection usually classified as “heuristic” but in the end is just reactive detection taken from a different angle. AV engines do have their place, but they’re not a solution — certainly not anymore. A small thought, and privacy advocates would hate this thought, is that maybe the AV vendors need to make their software 100%-report-to-base. Try to take some of the testing ability away from the criminals? Could this even be workable, what information could you report to base that’d help? How long would it be before the bad guys subverted the process or simply circumvented it… probably not long. sigh

I guess this is why the security industry is diversifying into more elements of command and control, maybe there is some light at the end of the tunnel? Of course is it likely that anything of this sort is best done at-or-below the OS level, thus by the OS vendor, but when Microsoft tried to do this for Vista there was an all-out cry of foul from the AV industry! Protecting themselves, or protecting users from the likelihood that Microsoft would get it wrong? A bit of both I expect.

In this direction lot of noise was made about one thing in the last year that to me smells like a load of of bollocks: virtualisation. It’s a very neat geek-toy that has spawned both it’s own industry around maintaining systems and has been co-opted by the security industry in a way that stinks of “silver-bullet”. The former works for me, but I think we want to keep in mind that virutalisation used this way is just an evolutionary step. Virtualisation for robustness/etc is a neat replacement for things like telnettable power supplies and Dell DRAC (remote administration) hardware. Security tends to be fitted in from a perspective of keeping an eye on things from the outside. We like this image because it works fairly well with physical-world security systems. My guess is that it isn’t going to work out quite as neatly or easily as hoped when it comes to anti-malware. I think the best anti-virutalisation FUD I’ve seen came from Theo, of OpenBSD fame.

[Update: In case it isn’t as blindly obvious as I thought, I agree with Theo de Raadt’s FUD (though I don’t understand why anybody thinks my agreement or labelling matters). sigh “FUD” is a just TLA, please attach less emotion to it Internet randoms. I’m wasting my time since the complaint I received was clearly derived purely from the sight of the TLA and the context ignored. Anyway, FUD = “Fear, Uncertainty, and Doubt” and in my mind is a mere function of marketing. Negative marketing based on perceived flaws in the security sphere is a case of FUD (since this is what it causes), sometimes for good (being informative), sometimes for bad (being misleading). Pro-virtualisation-for security people will label de Raadt’s opinion as FUD in the traditional sense, but I bag up what they see as smelly manure and feed it to my roses. I apologise for going against the grain of the TLA and upsetting a poor sensitive soul or two. To repeat: I, in my non-expert opinion, am more convinced by Theo’s FUD than the FUD from the other side of the argument. If it makes you feel better execute a mental s/FUD/marketing/g or just go away.]

Still, we have to grasp at what straws present themselves. (Remembering to let go of the ones that have burnt all the way down to our fingers.) I try to remind myself that entirely giving up hope is not the correct response. Especially while people are profiting from criminal acts that take advantage of the industry’s current failure to adequately deal with the problem.

At this moment, given a corporate network to run and short of “running with scissors”, I’d be focusing attention on environment control. Mostly meaning various approaches to controlled execution. I don’t think it’s an easy path, but does anyone expect a solution to really be “easy”? Hah! There’s a strong chance it’d just turn into another reactive scene, say we allow IE to run, fine, then malware runs it’s own code as part of IE. (Through one of virtually limitless vectors, from buffer buffer overflows inserting actual machine code to simple exploitation of design flaws in JS/VBS/Flash/plugin-X/technology-Y.) What about the much-maligned (at least it is in OSS/FSF circles) TPM approach? (Maybe just simplified virtualisation that’ll just come with a heap of it’s own new flaws.)

Network segregation should offer some relief and damage control. Do users really always need to access email/web from the same machine they access the IRS/HMRC/etc database from? At least if there is an infection (inevitable?) it can only go so far. This is heading into DLP territory though, which is a different problem and mostly the bugs that need to be fixed are in process and people.

Have we given up on user education yet? It’s bloody difficult, but I hope not. We can’t really expect people to always do the right thing, just as we can’t always expect programmers who know they should use validate all user data to always remember to do so (humans tend to be lazy by preference!). That said, the situation is certainly worse if they don’t even know what the right/wrong things are!

It’s easy to become despondent. I’m certainly not all that happy with the industry that I, in a small way, am part of. Taken as a whole the last year or two has been pretty abysmal. Surely things can only improve from this point?

A small hobby of mine to pick apart JavaScript/ECMA obfuscation such as that used by the Zhelatin/Storm/Nuwar “worm”. My usual approach, which is certainly inefficient, is to grok the actual code by translating it to Perl. I’ve written about this before in “Someone Doesn’t Like Kaspersky“.

I don’t usually have time, after wasting much in the process of grokking, to write about these critters and I don’t expect that to change much! Time is so hard to come by! But after looking at some of the code with recent Storm mailings I think it’s worth noting the evolution.

The previous obfuscation I’ve written about is simple application of “xor encryption”, and much of what I’ve seen has been a variation on this at a similar level of simplicity.

The basic xor case worked along the lines of the following pattern.

    function decode(A,B) {
        ...
        eval(C);
    }
    decode(ciphertext,key);

In this case the key (and thus ciphertext) value was randomly generated for different visits to the page. In the decode function B is applied byte-by-byte to A to gain the plaintext C. Usually this processing was xor (^) and was further complicated with a URI decode or something of that ilk.

The sample I have looked at most recently has the following form.

    function X(Y) {
        ...
        eval(Z);
    }
    X(payload);

The key differences are that the function name (X) is now a variable and the obvious key input is gone, which hints at something. What’s changed inside the code? Well, working from the final decrypt up to the start of the function, this is what happens (somewhat simplified, but this is the core pattern):

1. An array of 8 bytes is used as a key to shift the values in the input array in the manner of a classic Vigenère cipher applied mod-256).
2. The key array is obtained be encoding a 32 bit value (i.e. 2309737967) to hex (0x89ABCDEF) and using the ASCII value of each hex digit to populate the key array ([56, 57, 65, 66, 67, 68, 69, 70).
3. The 32 bit value is obtained by condensing an array of 256 integers (array256) and the text of the decode function (funcText) into an integer! The method iterates over characters in funcText using the byte values as lookup indexes in array256. Complete detail: key=0xFFFFFFFF; then for i in 0 to length(funcText) do:

   key=(array256[(key^funcText[i]) & 0xFF] ^ ((key >> 8) & 0xFFFFFF))

4. The text of the decode function is obtained with arguments.callee.toString(), which has non-word chars stripped out and is converted to all-caps. Thus the importance of the function name X as an input parameter to the obfuscation, it doesn’t stop there as the text for the rest of the function body is also part of this key material and is full of randomised variable names. As you may have guessed, is is the random function and variable names that change from one downloading of the script to another — rather than just the xor key.
5. The array of 256 integers is generated from a simple algorithm with a seed value, no need to detail it I think. It’s worth observing that between the different downloads of the script I saw the effective seed value didn’t change so this array remained constant.

Certainly much more complicated than the old xor code! But, I’d hope, a waste of time — since AV suspicious-script detection should work off generic patterns visible in the script from inspection rather than relying on the variable details. Still, only 3 AV engines on virustotal.com thought this script was worth noting as “generic obfuscated HTML”, but I don’t know what script/browser components they have enabled so I wouldn’t trust these out-of-context results. Many AV products exhibit different, usually more paranoid, behaviour when scanning in-browser data and HTTP at the gateway. And, looking at the whole Storm picture, this little snippet of code is just part of the delivery mechanism, it’s more important that the actual browser exploits and malware executables are caught!

Anyway, back to the script, this thing unwraps like a matryoshka doll. The plaintext is the same algorithm over again with new randomly generated function/variable names and a new ciphertext. The new ciphertext is much shorter though and after decoding we’re finished with this sample. The end result is javascript that generates a script DOM element and appends this to the document.

    var script = document.createElement("script");

    script.setAttribute("language", "JavaScript");
    script.setAttribute("src", "<nasty_local_url>");

    document.body.appendChild(script);

The most interesting item is the sample is this use of arguments.callee.toString() as key material. No doubt a direct defence against the usual malware-researcher practice of changing the final eval into an alert to expose the plaintext. While an admirable attempt at making life harder for researchers it’s not difficult to circumvent, just create a new variable assigned to the text “function X(Y) { ... }” and use this in place of the arguments.callee.toString() and good old alert should do it’s usual trick (then unwrap the next shell of the matryoshka). (Yes, “function” all that are included, though braces/punctuation don’t matter in the samples I have since an s/W//g is applied to the text)

The other “new technology” here is intriguing but not remarkable, using Vigenère instead of xor seems a curiosity more than a real advance (they’re certainly not doing it to hide the tell-tale use of the xor operator in a loop since they use xor in the key generation loops). Honestly, is looks just like some geek having fun, like me… but in this case we have a bad geek. Tut tut.

I’ve put a de-obfuscated and commented version of the script code up as well as a page containing active JavaScript that demonstrates the code. (Don’t worry, the active page’s payload is just an “alert” call!)

It’s been a while since I’ve had a Zhelatin/Storm/Nuwar mail get through to my inbox. Just in time for Christmas I get a shiny new one! It wishes me “Merry Christmas Dude” and provides a suitable URL for the season, no suspicious IP address link for this special occasion.

This one is a little different to previous efforts I’ve looked at. The embedded javascript isn’t malicious at all, in fact it is JSnow v0.2 complete with copyright notice. Snow! Joy! Is our favourite bot-net wishing us all a good Christmas out of good old fashioned social benevolence? Ha, fat chance! The page displays for us a set of scantily clad Mrs Clauses, enticing us to click on them for more. The link is to stripshow.exe, just less than 50% of the scanners on virustotal.com detect this at the moment. The list of ones that miss is conspicuously a round-up of the set with the largest market-share (interspersed with the ones that simply suck), this shouldn’t be any surprise these days.

It doesn’t stop there though, in a further effort the page embeds a javascript
in a I-Frame. And behold! We see the expected obfuscation code. So, in the end this isn’t really much different to previous sightings. I guess this strategy is still paying off for the crims behind it. It’s a sad indictment against the state of Internet security and security awareness that even after so many months this seemingly still works.

This time the javascript obfuscation is far more complex than others I’ve seen. Rather than a couple of simple translations we have several loops employing shifts and a variety of other bitwise operators (didn’t even know ECMA had an LSR operator). I guess they’ve invested some of their research time into this aspect of their code. At the moment only three of the virustotal.com scanners have anything to say about this and that’s just something along the lines of “generic obfuscated HTML”.

I wish people an infection-free Christmas. Have a good one.