Minimization tricks include:

• storing all sprites in a string by encoding each pixel in a sprite in 2 bits and storing 6 bits in each character. Each 2 bit chunk can have one of these values: 0=CR/lF (end of line/start of next line in sprite), 1=transparent (do not draw but move to next pixel), 2 = color 1, 3 = color 2. The colors are chosen from a palette, which is a string containing a number of RGB colors encoded in 3 hex chars (‘fff’=white, ’000′=black, ‘f00′=red, etc…). Each sprite has an associated palette based on the number of the sprite. Most sprites use one of two palettes: a one color palette (‘fff’) for the background stars) or a two color palette (’999000′) for the cat’s face, paws and tail. A few other palettes are used for the poptart.
• storing all sprite maps (meaning which sprites to draw where for each frame) in an array of 6 strings, one for each frame, by encoding each position and sprite number in 3 bits. The background stars are not included; their vertical position is based on their number, by multiplying the number of each star by a certain value. Their horizontal position is based on time, by multiplying the time by a certain value modulus the width of the image and adding a “random” value using the value of a different character from the sprites string for each star.
• drawing squares instead of sprites to fill larger regions, as sprites need a lot more bytes to store when they get bigger then code to draw a single square does.
• using one function for two purposes: to draw a specific sprite at a specific location using a specific palette on the canvas, or to draw a square at a specific location, with specific dimensions and a specific color on the canvas. The function will draw a sprite if there are only 4 arguments provided and a square if there are 5.
• using a bit of math to offset multiple sprites based on which frame is being drawn and adding sprites to the sprite. This is done to have the individual sprite maps for each frame contain the same information as much as possible, which makes them easier to compress.

I assumed it would be easy to code this one in 1K, but it turned out that it was pretty hard to minimize my code.

• I started with a large file that draw only the (animated) cat. This wasn’t optimized in any way: it used one big sprite for each frame, with each pixel being stored as byte containing a value 0-8, representing an index into a palette.
• I then cut the cat into smaller individual sprites that I could reuse between frames (eg. the face is always the same, just at a different location). I stored a sprite mapping, meaning which sprite to draw where for each frame. I stored all this information in a string using values between 0-36 encoded using ‘[char] = [value].toString(36)” and read using “[value] = parseInt([char], 36)”.
• I chose to give each sprite only two colors + transparent (3 values), so I can store each pixel in 2 bits (4 values) and use the unused value to signal ‘end of line’ and using ‘end of line’ twice for ‘end of sprite’. This meant I could store the entire sprite as 2 bit values. I encoded 3 of these 2 bit values per byte using “[char] = String.fromCharCode([values]+offset)” and red them using “[values] = [char].charCodeAt(0)-offset”.
• I stored the values in the sprite maps as 3 bit values and encoded 2 of these 3 bit values per byte as well.
• I wrote code that choses an ‘offset’ to use in encoding sprites/sprite maps that caused the resulting string to contain as many of the same bytes as the code, which allows better compression.
• At this point I was down to ~1.2K compressed. By reusing code for two purposes (eg. the ‘f’ function) and rearranging/rewriting code to have it contain as many repeated strings as possible to allow better compression, I as able to get it down to ~1.01K.
• Finally, I optimized my png compressor code to work together with the poptart cat code; eg. do not use ‘eval’ to run the decompressed code, which then uses setTimeout to start the animation, but use ‘setTimeout’ directly. This allowed me to squeeze everything into 1K.

Because manually editing 2/3 bit values stored in 8 bit characters + offset is impractical, and because some of the values where calculated rather than hand-picked. I wrote my development code to generate the final code and data automatically. This means I have a somewhat documented, readable version of the test container HTML and the actual JavaScript code. The test container will load the code, which will show the animation slowed down to 1 frame per second and output the final code below the animation. The final code is what I pumped through my to-be-released JsSfx png compressor.

Afterwards, I realized that there are a few ways in which I could have improved on the code by doing a few things differently:

• Save data as unicode strings; this allows storing not 6 bits per byte, but (effectively) 16 per 2 bytes. This could reduce the size of the data by about 128 bytes, but would use more bytes to store the non-unicode characters outside of the strings.
• Save each entry for each sprite map in a separate string, meaning store the x coordinates in one string, the y coordinates in another and the sprite number in a third. This should result in data that can be compressed better.
• Use scaling for sprites to be able to draw the background, rainbow and body of poptart cat using sprites as well instead of separate code to draw squares. This would use less bytes than having code to draw the cat using both sprites and squares.

Suggestions and comments are appreciated; I have a few ideas of my own, such as allowing you to link to specific zoom settings, but I’m not sure when I’ll have time to add them.

Source code here, install it here

The size of the shellcode is 242 bytes (add 5 for stack alignment and 39 for EAT bypass). It has all the usual bells and whistles: OS/SP independent, null-free, optional stack alignment and EAT bypass and no register requires a specific value for it to run correctly.

Get the code here.

JsSfx is designed for use in the demo scene, where people often create JavaScript demos in a very limited number of bytes (and often a power of 2, such as 256 bytes, 512 bytes, 1Kb, 4Kb, etc…). In August/September of 2010, a demo competition was held at js1k.com. Contestants were asked to submit JavaScript demos that were 1024 bytes or less in size. A few of the entries used compression to fit a larger script into 1k. One of the more complex entries, WOLF1k by p01, uses cowboy‘s packify to compress 1370 bytes of script into exactly 1024 bytes. To see how well JsSfx was doing, I tested JsSfx1.2 on the 1370 uncompressed bytes of WOLF1k and found it reduced it to 1031 bytes. Obviously, I needed to improve on that, so I came up with an improved decoder for JsSfx3. The latest version compresses WOLF1k to 1013 bytes in utf-8 and 986 bytes using ISO 8859-1 (aka. latin-1). However, it does take a while to find the best way to compress scripts. JsSfx3 was developed with size in mind and not speed: it takes a minute or two to compress scripts of a few kilobytes. Compressing large scripts such as jQuery is not practical: it would probably take days on a fast computer.

There’s another competition over at js1k.com at the moment, but unfortunately compression is not allowed this time. Peter van der Zee, who hosts the contest, seems to believe that using compression removes the need for “hand tuning” your script and that this removes the fun. I disagree and believe WOLF1k and my own Perlin Noise script prove that. Also, I can assure you that a lot of “hand tuning” went into JsSfx3.2.

Anyway, if you’re working on small JavaScript demos and would like to see how small you can get your script, please give the new version a try and let me know if it works for you!

I still believe it is a good exercise to extract the source from the encoded version, as you learn a lot more that way, but I’ll leave that choose to you.

One of my side projects is creating JavaScript demos. Unfortunately, I hardly ever find time to finish and release anything. You may recognize the following: I think of a new effect that would be really cool. I start working on some code and get a proof-of-concept working. The code proves how awesome the end result would be, but it’s nowhere near production quality. Then I waste a lot of time on fine-tuning some aspects of the code (size, speed, various design/implementation details). Finally I get bored with it and/or distracted by a new project. I have a zillion of such projects doing nothing on my harddrive.

Luckily, this is not always the case. I was going through some of the awesome codes of the js1k contest recently and got interested in Perlin noise. I started working on my own version and after only a few iterations I had a pretty decent flame effect going. The amazing thing about Perlin noise is how easy it is to create a decent looking effect with very little effort.

The code starts by creating an array with random numbers. The code needs to have a way of requesting the same random number twice and getting the same value and an array is the fasted way to do this. Using an index into the array as a seed, the code can read the same random number as often as it wants without changes. One side effect is that because the size of the array is limited, you will at some point have to reuse the same data. However, if you make sure the size of this array is sufficiently large (and a prime), this will not be noticed by the average user. Next the code creates three canvas elements. Two are used as off-screen buffers to generate the flames and smoke in, the third is visible and used to combine the two into the final effect. Two one dimentional arrays of perlin data are generated, one for the shape of the flames and one which is used as a projection map for the movement of the flames. For each pixel, the projection map tells the code where in the shape map it should read a byte to use as the color value for that pixel. The color value is converted to an RGB value using two table, one for the flames and one for the smoke. The pixels are drawn at the bottom of the off-screen buffers and all pixels in these buffers are shifted upwards. For the flames, all pixels in the buffer are darkened, causing them to die out as the move up.

When I got his working, I got obsessed, as one does, with size and speed: I wanted to have it run at 60fps and shrink the size of the code to 1k or less. Guaranteeing the frame rate is hard to achieve because the effect needs a few CPU intensive operations to calculate each frame. I could reduce the quality to achieve a higher frame rate, but that would defeat the original goal of creating a nice looking effect. After a bit of tweaking, it is now running at 60fps on my machine without too much loss of quality. However, it may be less on older machines.

Shrinking the size of JavaScript is mostly about rewriting/rearranging your algorithms to be more size efficient. How to best do this depends on the algorithm and many such optimizations are one-offs: they apply only to that one code construct and you’ll probably never use the same trick again. However, there are a few tricks that I find I can commonly use on all scripts to reduce their size:

1. Use one character, global variable and function names:
   • "var iCounter = 0" => "i=0"
2. Do not use expensive keywords:
   • "x=new Array();" => "x=[];"
   • "while(){}", "do {} while ()" => "for(){}"
   • "x=Math.floor(x);" => "x=x>>0;"
   • "x=Math.round(x);" => "x=x-.5>>0;"
   • "x=Math.ceil(x);" => "x=x+.999>>0;"
     (if you don’t mind rounding errors)
   • "x=Math.pow(2,x);" => "x=1<<x;"
   • "x=x/256;" => "x=x>>8;"
     (if you don’t mind the result being rounded down to an integer)
3. Choose the most efficient notation for numbers:
   • "0×10" => "16"
   • "0×20000" => "1<<17"
   • "1000" => "1e3"
   • ".0001" => "1e-4"
4. Optimize how you use the three-expression in for-loops:
   • "for(x=0;x<50;x++){}" => "for(x=50;x–;){}"
   • "for(x=50;x–;){}y=5;" => "for(x=50;x–;y=5){}"
     (this saves 1 semi-colon, but may slow your script down a little)
5. Remove comments and unneeded curly braces, white-space and semi-colons:
   • "for (…) { a+=b; c*=a; }" => "for(…)a+=b,c*=a;"
   • "function () {a+=b;}" => "function(){a+=b}"
   (Some of this can be done automatically and I’ve recently added code to do this to JsSfx)
6. Save often used complex values in variables:
   • "x=document.createElement(…);document.body.appendchild(x);" => "d=document;x=d.createElement(…);d.body.appendchild(x);"
   • "y=x*x*x+x*x-4;z=x*x*x+x*x+5;" => "y=(q=x*x*x+x*x)-4;z=q+5;"
7. Combine setting values:
   • "x=0;y=0;" => "x=y=0;"
   • "x=0;y=[0];" => "y=[x=0];"

This list does not claim to be complete. If you feel that I have omitted something useful, please leave a comment.

After manual optimization of code size, you can probably shrink the size of you script a lot further using some form of compression and self-extraction. I published JsSfx, which implements a custom compression and generates self-extracting JavaScript. I developed the compression technique it uses based on a number of assumptions:

1. JavaScript often contains the same character sequence a number of times (eg. strings like "for(" will appear quite often)
2. JavaScript normally uses a small sub-set of all the bytes valid in JavaScript.
3. The size of the script to be compressed is assumed to be small. Therefore, compression can save only so many bytes and the size of the decoder will have a large impact on the size of the final script. A large decoder negates the benefit of a good compression ratio.

The compression used in JsSfx works by replacing all instances of an often repeated string with one character that is not used in the script. It saves the character and the string it replaced as well. In a way this is similar to the manual step number 6 I explained above. Here’s an example:

code="o = document.createElement('a');\r\ndocument.body.appendChild(o);"
 

May be compressed as:

keys="A"
code="o = AcreateElement('a');\r\nAbody.appendChild(o);Adocument.";
 

In the above example, “document.” was replaced by “A”. The keys variable contains the list of the characters used to replace strings (in this case only “A”). The code variable stores the original string, with the replacements, followed by the key and the replaced string. The original code can be restored by splitting the code string into sub-strings using the keys. In the above example this leads to an array of sub-strings like this:

sub_string=["o = ", "createElement('a');\r\n", "body.appendChild(o);", "document."];
 

By removing the last string from the array and joining the remaining strings back together using this string, the decoder can recreate the original.

The original Perlin noise flames script was about 3.2k in size (excluding comments and whitespace). I manually reduced this to ~1.2k (a lot of gain comes from replacing long variables names with one character). Using JsSfx, I got it just below 1k.

I’d like to thank the people at pouet.net for feedback that allowed me to get this script to run on all browsers except MSIE. I’ve tested it on MSIE9 platform preview, but it doesn’t work (for unknown reasons).

In early September this year Microsoft released their Enhanced Mitigation Experience Toolkit v2.0 (EMET), which includes a new “pseudo”-mitigation called Export address table Address Filter (EAF). I decided to have a look at how this mitigation attempts to prevent exploits from succeeding and how an attacker might bypass it. For people that suffer from tl;dr syndrome, I’ve put my conclusion up front:

It is my conclusion that EAF should be effective at preventing most current shellcode from executing and therefore a useful mitigation. However, it is relatively simple to bypass. Proof of concept code to do this can be found here. I expect that if EAF becomes a common mitigation, attackers will update their shellcodes to bypass it. I cannot think of any effective way in which EAF can be updated that would not be relatively simple to bypass as well.

EAF works by setting a hardware breakpoint on the export address tables of the ntdll.dll and kernel32.dll modules in a process. When the breakpoint is triggered, EAF determines if the code that is trying to access the export address table is valid code for that process or malicious code injected into the process through an exploit. Most exploits will at some point inject and run shellcode into the target process. One of the first thing most shellcodes do is determine where certain functions are loaded in memory. This is commonly and easiest done by going through the list of loaded modules and reading their export address tables. When shellcode reads the export address tables of ntdll.dll and/or kernel32.dll, EAF detects the shellcode and terminates the process, preventing the exploit from running successfully.

I tested EAF withone of my shellcodes that shows a message in a popup window when it runs successfully. All my shellcodes scan the export address tables of loaded modules to find certain functions. This is a common technique used by almost all shellcodes that EAF is designed to detect. So, enabling EAF should prevent my shellcode, and most others, from working.

I use testival to test my shellcode because it makes things easy to automate and I can copy+paste the output into this blog to show you what is going on. When my shellcode is run, it shows a popup dialog box and then triggers an int3 debugger breakpoint. Here’s the output of w32-testival.exe for a successful run of my shellcode:

C:\Dev\Shellcode\w32-msgbox-shellcode>w32-testival [$]=ascii:w32-msgbox-shellcode.bin eip=$ --verbose --eh --eh
Allocating 0x1000 bytes of memory... ok. (address: 0x00030000)
Setting data and registers:
[0x00030000] = 8C bytes of data.
eax = 0xDEADBEEF (default)
ecx = 0xDEADBEEF (default)
edx = 0xDEADBEEF (default)
ebx = 0xDEADBEEF (default)
esp = ??? (unmodified)
ebp = 0xDEADBEEF (default)
esi = 0xDEADBEEF (default)
edi = 0xDEADBEEF (default)
eip = 0x00030000 ($)
Registering Structured Exception Handler (SEH)...ok.
Registering Vectored Exception Handler (VEH)...ok.
Executing shellcode by jumping to 0x00030000...First chance debugger breakpoint exception at 0x0003008B.
Second chance debugger breakpoint exception at 0x0003008B.
 
C:\Dev\Shellcode\w32-msgbox-shellcode>
 

Of course, you cannot see the popup dialog box in this output, but you can see that the int3 debugger breakpoint at the end of the shellcode was executed. This means that it ran successfully.

After enabling EAF I tried executing the shellcode again and found that it still worked. So either EAF was not working or I was doing something wrong. I contacted my friends at MS who developed the tool and asked them to help me find out what was going on. They explained that in order to install the EAF mitigation, EMET needs to create a new thread in the process first, which means the mitigation is not enabled immediately on startup. To make sure that the mitigation is installed, you need to wait a bit before running your shellcode. So, I added a new switch to testival that allows it to wait a given number of milliseconds before executing the shellcode. Using this new feature, I tried again and this time EAF successfully blocked my shellcode, as you can see here:


C:\Dev\Shellcode\w32-msgbox-shellcode>w32-testival [$]=ascii:w32-msgbox-shellcode.bin eip=$ --verbose --eh --eh --delay=1000
Allocating 0x1000 bytes of memory... ok. (address: 0x00030000)
Setting data and registers:
[0x00030000] = 8C bytes of data.
eax = 0xDEADBEEF (default)
ecx = 0xDEADBEEF (default)
edx = 0xDEADBEEF (default)
ebx = 0xDEADBEEF (default)
esp = ??? (unmodified)
ebp = 0xDEADBEEF (default)
esi = 0xDEADBEEF (default)
edi = 0xDEADBEEF (default)
eip = 0x00030000 ($)
Registering Structured Exception Handler (SEH)...ok.
Registering Vectored Exception Handler (VEH)...ok.
Waiting for 1000 milliseconds...ok.
Executing shellcode by jumping to 0x00030000...First chance single step exception at 0x00030054: A trace trap or other single-instruction mechanism signaled that one instruction has been executed.
Second chance exception 0xC0000409 at 0x00030054.
 
C:\Dev\Shellcode\w32-msgbox-shellcode>
 

This time the process is terminated early by a single step exception. This is because the EAF mitigation has detect that the shellcode accessed the export address table and decided to terminate the application rather than allow the shellcode to continue executing. This shows that EAF should be able to detect and protect against exploits that use most common shellcode.

I decided not to reverse EAF in order to find out how it works, but rather try to guess how it works and guess how it might by bypassed and try if it works. I assumed that EAF works by checking the location of the instruction that is accessing the export address table: if it is located inside the code segment of a loaded module, EAF assumes it is valid code and allowed it to continue. Otherwise, EAF assumes it is malicious code and terminates the process. If my hypothesis is correct, it might be possible to have the shellcode use a sequence of instructions inside the code segment of a loaded module that can be used to read memory. Because in this case, EAF will assumes that valid code is attempting to read the export address table rather than my shellcode and allow the code to continue.

To test this, I created a modified version of my shellcode that works like this:

• First, the shellcode finds out where ntdll.dll is loaded in memory as usual.
• Second, it finds out where the code segment for ntdll.dll is located.
• Third, it scans the code segment for a specific instruction sequence that can be used to read arbitrary memory.
• Finally, it calls this instruction sequence to read the export address table, rather than read it directly.

This is effectively the same as doing a ret-into-libc attack, something which EAF is not designed to block and this should therefore be able to bypass it. It turns out that the RtlGetCurrentPeb function has a useful instruction sequence. This sequence is static across Windows versions and SPs and exactly 4 bytes long, which means it is easy to write code to find it:

ntdll32!RtlGetCurrentPeb:
64a118000000 mov eax,dword ptr fs:[00000018h]
8b4030 mov eax,dword ptr [eax+30h]
c3 ret
 

By setting EAX to the memory address you want to read (minus 0×30) and calling the second instruction, you can read arbitrary memory into EAX.

The first time I tested my code, it was blocked by EAF while scanning for the instruction sequence in the code segment. It turns out that the export address table is located at the start of the code segment of ntdll.dll, so my scan for the instruction sequence was accessing it and triggering EAF. Luckily, the RtlGetCurrentPeb function is not located anywhere near the start of the code segment in any version of ntdll.dll, so it was relatively easy to avoid this by skipping over the first 0×1000 bytes of the code segment.

Here is the result for my modified shellcode, which is only 30 bytes larger than the original:

C:\Dev\Shellcode\w32-msgbox-shellcode>w32-testival [$]=ascii:w32-msgbox-shellcode-eaf.bin eip=$ --verbose --eh --eh --delay=1000
Allocating 0x1000 bytes of memory... ok. (address: 0x00030000)
Setting data and registers:
[0x00030000] = AB bytes of data.
eax = 0xDEADBEEF (default)
ecx = 0xDEADBEEF (default)
edx = 0xDEADBEEF (default)
ebx = 0xDEADBEEF (default)
esp = ??? (unmodified)
ebp = 0xDEADBEEF (default)
esi = 0xDEADBEEF (default)
edi = 0xDEADBEEF (default)
eip = 0x00030000 ($)
Registering Structured Exception Handler (SEH)...ok.
Registering Vectored Exception Handler (VEH)...ok.
Waiting for 1000 milliseconds...ok.
Executing shellcode by jumping to 0x00030000...First chance single step exception at 0x76FBA045: A trace trap or other single-instruction mechanism signaled that one instruction has been executed.
First chance single step exception at 0x76FAFFCE: A trace trap or other single-instruction mechanism signaled that one instruction has been executed.
First chance single step exception at 0x76FAFFCE: A trace trap or other single-instruction mechanism signaled that one instruction has been executed.
First chance single step exception at 0x76FAFFCE: A trace trap or other single-instruction mechanism signaled that one instruction has been executed.
<snip>(Many more single step exceptions)<snip>
First chance single step exception at 0x76FAFFCE: A trace trap or other single-instruction mechanism signaled that one instruction has been executed.
First chance debugger breakpoint exception at 0x000300AA.
Second chance debugger breakpoint exception at 0x000300AA.
 
C:\Dev\Shellcode\w32-msgbox-shellcode>
 

Every time the export address table is accessed, the hardware breakpoint is triggered and EAF checks to see if shellcode is attempting to access it. This is why you are seeing so may first change single step exceptions. However, because I am using the instruction sequence in ntdll.dll to access the address table, EAF allows the code to continue. The shellcode runs successfully and the process terminates when the shellcode executes the int3 debugger breakpoint as before.

Have a look at the changes I made to my shellcode to make this work. They are relatively minor and it should be possible to apply this technique to any shellcode, which reduces the usefulness of EAF in the long run.

I thought about possibilities to prevent or detect this bypass. Unfortunately, I could not think of anything that would be effective. Here are some of my ideas and why I think they won’t work:
- EAF could check for this specific instruction sequence. However, shellcode could scan for other sequences in ntdll.dll (or even in other modules) that can be used to achieve the same.
- EAF could “walk the stack” and check that all return addresses are valid. However, the shellcode could construct a ret-into-libc stack with only valid return addresses, where the first call reads the export address table and the next call modifies the return address for the third return address to point to the shellcode again.
- EAF could set additional hardware breakpoints on structures that can be used to find the locations of modules, in an attempt to prevent the shellcode from finding an instruction sequence that it can use to read memory. However, there are a large number of ways in which the location of modules can be found and there are a limited number of hardware breakpoints. There are not enough breakpoints available to protect all of locations that contain sensitive data.

I welcome your thoughts and ideas on this subject.