Malware analysis: part 12. Section entropy from scratch in pure C: a tiny packed/encrypted detector.
﷽
Hello, cybersecurity enthusiasts and white hackers!
In part 6 we first met Shannon entropy, and in part 11 we built a whole mini-GPT to detect packing by perplexity. Both were Python and, in the GPT case, a GPU. In this post we go in the exact opposite direction: the smallest, most portable blue team triage tool I could think of - section entropy in pure C, one file, zero dependencies (only libc + libm), that compiles in a single line and runs on both PE and ELF.
The idea is that you can drop this on any incident-response box - a stripped Linux server, a rescue shell, a customer laptop - compile it in one command and instantly see which section of a suspicious binary looks packed or encrypted. No pip, no venv, no cloud API. I called it pe-kitten and the tool itself is meow.c.
the idea
A binary file is just a sequence of bytes. Compression and encryption both do the same thing to those bytes: they make them look like uniform random noise. Legitimate code and data are highly structured - opcodes repeat, strings cluster, padding is full of zeros - so their byte distribution is far from uniform. That difference is exactly what entropy measures.
The right granularity is per section, not per file. A packer leaves a very recognizable footprint: a small low-entropy loader stub plus one section whose entropy is pushed close to the theoretical maximum (the compressed/encrypted body). Averaged over the whole file this can hide; looked at section by section it screams.
This is conceptually the “dumb” cousin of the perplexity detector from part 11: perplexity models the order of bytes, entropy only counts their frequencies. But entropy is \(O(n)\), needs no model, no training and no dependencies - which is precisely what you want for a first-pass triage tool that has to run anywhere.
the math
The Shannon entropy of a section, measured over its 256 possible byte values, in bits per byte:
\[H = -\sum_{i=0}^{255} p_i \log_2 p_i, \qquad p_i = \frac{\mathrm{count}_i}{N}\]where \(N\) is the section size in bytes and \(\mathrm{count}_i\) is how many times byte value \(i\) appears. \(H\) ranges from 0 (a section of one repeated byte) to 8 (all 256 values equally likely - perfectly random). Encryption and good compression drive \(H\) toward 8, which is the whole reason packed samples stand out.
How high is “suspicious”? We do not have to guess. Lyda & Hamrock measured the average section entropy across large corpora of real files:
| content | mean \(H\) (bits/byte) |
|---|---|
| plain text | 4.35 |
| native executable | 5.10 |
| packed | 6.80 |
| encrypted | 7.17 |
So meow uses two thresholds: below 6.0 is green (normal), 6.0 - 7.0 is yellow (elevated - typical of dense native .text), and >= 7.0 is red - likely packed or encrypted.
practical example
Let’s build meow.c piece by piece. The heart of the tool is the entropy function - a 256-bucket histogram and the sum above:
/* shannon entropy of a byte buffer, in bits/byte (0..8).
* h = -sum_i p_i * log2(p_i), p_i = count_i / n (shannon, 1948). */
static double entropy(const uint8_t *data, uint64_t n) {
if (!n) return 0.0;
uint64_t hist[256] = {0};
for (uint64_t i = 0; i < n; i++) hist[data[i]]++;
double h = 0.0;
for (int i = 0; i < 256; i++) {
if (!hist[i]) continue;
double p = (double)hist[i] / (double)n;
h -= p * log2(p);
}
return h;
}
That is the whole math. Everything else is just locating the section bytes in the file. We support both formats and detect them by magic in main:
if (fsize >= 2 && buf[0] == 'M' && buf[1] == 'Z')
rc = analyze_pe(buf, fsize);
else if (fsize >= 5 && !memcmp(buf, "\x7f" "ELF", 4))
rc = analyze_elf(buf, fsize);
For PE we walk exactly the offsets from part 10: the DOS header points to the NT headers via e_lfanew at 0x3C, then we read NumberOfSections and skip the optional header by its declared size to land on the 40-byte section headers. Each header gives us SizeOfRawData and PointerToRawData - the slice we feed to entropy():
uint32_t e_lfanew = rd32(buf + 0x3C);
const uint8_t *coff = buf + e_lfanew + 4; /* skip "PE\0\0" */
uint16_t nsec = rd16(coff + 2); /* NumberOfSections */
uint16_t optsz = rd16(coff + 16); /* SizeOfOptionalHeader */
const uint8_t *sect = coff + 20 + optsz; /* section table start */
for (uint16_t i = 0; i < nsec; i++) {
const uint8_t *sh = sect + (uint64_t)i * 40;
char name[9]; memcpy(name, sh, 8); name[8] = 0;
uint32_t rawsz = rd32(sh + 16); /* SizeOfRawData */
uint32_t rawptr = rd32(sh + 20); /* PointerToRawData */
double h = 0.0;
if (rawsz && (uint64_t)rawptr + rawsz <= fsize) h = entropy(buf + rawptr, rawsz);
print_section(name, rawsz, h);
}
For ELF64 it is the same story with different field names: e_shoff gives the section-header table, e_shstrndx tells us which section holds the name strings (.shstrtab), and each 64-byte header carries sh_offset and sh_size. We skip SHT_NOBITS (.bss has no bytes on disk):
uint64_t shoff = rd64(buf + 0x28);
uint16_t shentsize = rd16(buf + 0x3A);
uint16_t shnum = rd16(buf + 0x3C);
uint16_t shstrndx = rd16(buf + 0x3E);
const uint8_t *strsh = buf + shoff + (uint64_t)shstrndx * shentsize;
uint64_t stroff = rd64(strsh + 0x18); /* .shstrtab file offset */
for (uint16_t i = 0; i < shnum; i++) {
const uint8_t *sh = buf + shoff + (uint64_t)i * shentsize;
uint32_t sh_name = rd32(sh + 0);
uint32_t sh_type = rd32(sh + 4);
uint64_t sh_off = rd64(sh + 0x18);
uint64_t sh_size = rd64(sh + 0x20);
if (sh_type == 8 /* SHT_NOBITS */ || sh_size == 0) continue;
const char *name = (const char *)(buf + stroff + sh_name);
double h = (sh_off + sh_size <= fsize) ? entropy(buf + sh_off, sh_size) : 0.0;
print_section(name, sh_size, h);
}
Finally the pretty part - print_section draws a colored ASCII bar whose length is \(H/8\) of the width, and tags anything over the thresholds:
static void print_section(const char *name, uint64_t size, double h) {
const int W = 32;
int fill = (int)(h / 8.0 * W + 0.5);
const char *col = ent_color(h); /* green / yellow / red */
printf(" " C_CYN "%-16.16s" C_RST " %10llu %s%5.3f" C_RST " %s",
name, (unsigned long long)size, col, h, col);
for (int i = 0; i < W; i++) putchar(i < fill ? '#' : ' ');
printf(C_RST);
if (h >= ENT_HIGH) printf(" " C_RED C_BLD "packed/encrypted?" C_RST);
else if (h >= ENT_ELEVATED) printf(" " C_YEL "high" C_RST);
putchar('\n');
}
The full meow.c (with bounds checks and the file loader) is about 230 lines:
/* pe-kitten / meow.c - tiny pe/elf section entropy analyzer
* author: cocomelonc
*
* reads a pe (mz) or elf binary, walks its section table by the exact
* on-disk offsets, and reports the shannon entropy of each section's raw
* bytes with an ascii color bar. high entropy flags likely packed or
* encrypted content (lyda & hamrock, ieee s&p 2007).
*
* deps: libc + libm only.
* build: gcc -O2 -Wall -Wextra -std=c11 -o meow meow.c -lm
* usage: ./meow <file>
*/
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
/* ---- ansi colors (plain vt100) ---------------------------------------- */
#define C_RST "\x1b[0m"
#define C_DIM "\x1b[2m"
#define C_BLD "\x1b[1m"
#define C_GRN "\x1b[32m"
#define C_YEL "\x1b[33m"
#define C_RED "\x1b[31m"
#define C_CYN "\x1b[36m"
/* lyda & hamrock 2007 measured mean entropies (bits/byte):
* plain text 4.35, native exe 5.10, packed 6.80, encrypted 7.17.
* we use 6.0 / 7.0 as the elevated / high thresholds. */
#define ENT_HIGH 7.0
#define ENT_ELEVATED 6.0
/* alignment-safe little-endian reads (x86 target). */
static uint16_t rd16(const uint8_t *p) { return (uint16_t)(p[0] | (p[1] << 8)); }
static uint32_t rd32(const uint8_t *p) { uint32_t v; memcpy(&v, p, 4); return v; }
static uint64_t rd64(const uint8_t *p) { uint64_t v; memcpy(&v, p, 8); return v; }
/* shannon entropy of a byte buffer, in bits/byte (0..8).
* h = -sum_i p_i * log2(p_i), p_i = count_i / n (shannon, 1948). */
static double entropy(const uint8_t *data, uint64_t n) {
if (!n) return 0.0;
uint64_t hist[256] = {0};
for (uint64_t i = 0; i < n; i++) hist[data[i]]++;
double h = 0.0;
for (int i = 0; i < 256; i++) {
if (!hist[i]) continue;
double p = (double)hist[i] / (double)n;
h -= p * log2(p);
}
return h;
}
/* pick a color for an entropy value. */
static const char *ent_color(double h) {
if (h >= ENT_HIGH) return C_RED;
if (h >= ENT_ELEVATED) return C_YEL;
return C_GRN;
}
/* one section row: name, size, entropy value + colored bar + verdict. */
static void print_section(const char *name, uint64_t size, double h) {
const int W = 32;
int fill = (int)(h / 8.0 * W + 0.5);
const char *col = ent_color(h);
printf(" " C_CYN "%-16.16s" C_RST " %10llu %s%5.3f" C_RST " %s",
name, (unsigned long long)size, col, h, col);
for (int i = 0; i < W; i++) putchar(i < fill ? '#' : ' ');
printf(C_RST);
if (h >= ENT_HIGH) printf(" " C_RED C_BLD "packed/encrypted?" C_RST);
else if (h >= ENT_ELEVATED) printf(" " C_YEL "high" C_RST);
putchar('\n');
}
/* running max entropy across sections, for the final verdict. */
static double g_max_ent = 0.0;
static void account(double h) { if (h > g_max_ent) g_max_ent = h; }
/* ---- pe ---------------------------------------------------------------- */
static int analyze_pe(const uint8_t *buf, uint64_t fsize) {
if (fsize < 0x40) return 1;
uint32_t e_lfanew = rd32(buf + 0x3C);
if ((uint64_t)e_lfanew + 24 > fsize || memcmp(buf + e_lfanew, "PE\0\0", 4)) {
fprintf(stderr, "meow: not a valid pe (bad NT signature)\n");
return 1;
}
const uint8_t *coff = buf + e_lfanew + 4;
uint16_t nsec = rd16(coff + 2);
uint16_t optsz = rd16(coff + 16);
const uint8_t *sect = coff + 20 + optsz; /* section table start */
printf(C_BLD "PE" C_RST " image, %u section%s\n\n", nsec, nsec == 1 ? "" : "s");
printf(" " C_DIM "%-16s %10s %5s %-32s" C_RST "\n",
"name", "raw size", "H", "entropy (bits/byte)");
for (uint16_t i = 0; i < nsec; i++) {
const uint8_t *sh = sect + (uint64_t)i * 40;
if ((uint64_t)(sh + 40 - buf) > fsize) break;
char name[9]; memcpy(name, sh, 8); name[8] = 0;
uint32_t rawsz = rd32(sh + 16);
uint32_t rawptr = rd32(sh + 20);
double h = 0.0;
if (rawsz && (uint64_t)rawptr + rawsz <= fsize) h = entropy(buf + rawptr, rawsz);
account(h);
print_section(name, rawsz, h);
}
return 0;
}
/* ---- elf (64-bit little-endian) --------------------------------------- */
static int analyze_elf(const uint8_t *buf, uint64_t fsize) {
if (buf[4] != 2 /* ELFCLASS64 */ || buf[5] != 1 /* ELFDATA2LSB */) {
fprintf(stderr, "meow: only elf64 little-endian supported for now\n");
return 1;
}
uint64_t shoff = rd64(buf + 0x28);
uint16_t shentsize = rd16(buf + 0x3A);
uint16_t shnum = rd16(buf + 0x3C);
uint16_t shstrndx = rd16(buf + 0x3E);
if (shoff + (uint64_t)shnum * shentsize > fsize) {
fprintf(stderr, "meow: elf section table out of range\n");
return 1;
}
/* .shstrtab: string table holding section names. */
const uint8_t *strsh = buf + shoff + (uint64_t)shstrndx * shentsize;
uint64_t stroff = rd64(strsh + 0x18);
printf(C_BLD "ELF64" C_RST " image, %u section%s\n\n", shnum, shnum == 1 ? "" : "s");
printf(" " C_DIM "%-16s %10s %5s %-32s" C_RST "\n",
"name", "size", "H", "entropy (bits/byte)");
for (uint16_t i = 0; i < shnum; i++) {
const uint8_t *sh = buf + shoff + (uint64_t)i * shentsize;
uint32_t sh_name = rd32(sh + 0);
uint32_t sh_type = rd32(sh + 4);
uint64_t sh_off = rd64(sh + 0x18);
uint64_t sh_size = rd64(sh + 0x20);
if (sh_type == 8 /* SHT_NOBITS (.bss) */ || sh_size == 0) continue;
const char *name = (const char *)(buf + stroff + sh_name);
double h = 0.0;
if (sh_off + sh_size <= fsize) h = entropy(buf + sh_off, sh_size);
account(h);
print_section(name, sh_size, h);
}
return 0;
}
/* slurp a whole file into a malloc'd buffer. */
static uint8_t *read_file(const char *path, uint64_t *out) {
FILE *f = fopen(path, "rb");
if (!f) { perror(path); return NULL; }
fseek(f, 0, SEEK_END);
long sz = ftell(f);
fseek(f, 0, SEEK_SET);
if (sz <= 0) { fclose(f); fprintf(stderr, "%s: empty\n", path); return NULL; }
uint8_t *buf = malloc((size_t)sz);
size_t got = buf ? fread(buf, 1, (size_t)sz, f) : 0;
fclose(f);
if (!buf || got != (size_t)sz) { free(buf); return NULL; }
*out = (uint64_t)sz;
return buf;
}
int main(int argc, char **argv) {
if (argc != 2) {
fprintf(stderr, "pe-kitten / meow (author: cocomelonc)\n"
"usage: %s <pe-or-elf-file>\n", argv[0]);
return 2;
}
uint64_t fsize = 0;
uint8_t *buf = read_file(argv[1], &fsize);
if (!buf) return 1;
printf(C_BLD "meow" C_RST " :: %s (%llu bytes)\n whole-file H = %s%.3f" C_RST
" bits/byte\n\n", argv[1], (unsigned long long)fsize,
ent_color(entropy(buf, fsize)), entropy(buf, fsize));
int rc;
if (fsize >= 2 && buf[0] == 'M' && buf[1] == 'Z')
rc = analyze_pe(buf, fsize);
else if (fsize >= 5 && !memcmp(buf, "\x7f" "ELF", 4))
rc = analyze_elf(buf, fsize);
else {
fprintf(stderr, "meow: unknown format (need MZ/pe or \\x7fELF)\n");
rc = 1;
}
if (rc == 0) {
printf("\n verdict: max section H = %s%.3f" C_RST " -> %s\n",
ent_color(g_max_ent), g_max_ent,
g_max_ent >= ENT_HIGH
? C_RED C_BLD "likely packed/encrypted" C_RST
" (lyda & hamrock 2007 threshold ~7.0)"
: C_GRN "no packed/encrypted section detected" C_RST);
}
free(buf);
return rc;
}
demo
First of all, for compilation we need any Linux box with a C compiler:
gcc -O2 -Wall -Wextra -std=c11 -o meow meow.c -lm
That is the entire toolchain. -lm is only there for log2(). The same source builds with clang or with mingw if you want a Windows binary.
Let’s point it at a known-good ELF first - /bin/ls:
./meow /bin/ls
Everything is green except .text, which sits around 6.3 and is tagged high - completely normal for dense compiled x86 code, and a good reminder that “elevated” is not the same as “malicious”. The verdict line reports the maximum section entropy and concludes no packed section was found.
Now a real PE. I used one of the Ghostpack binaries lying around, Rubeus.exe:
./meow Rubeus.exe
Three sections - .text, .rsrc, .reloc - all comfortably below the packed threshold. This is an ordinary (unpacked) .NET assembly, and meow says so.
demo 2
To see the red path we need something actually packed. No UPX on the box? We can fake a packer honestly in two commands: take a clean mingw-built hello.exe, find its .text file offset with objdump, and overwrite that section with random bytes - exactly what a crypter’s body looks like:
cp hello.exe packed.exe
read OFF SZ < <(objdump -h packed.exe | awk '$2==".text"{print strtonum("0x"$6), strtonum("0x"$3)}')
dd if=/dev/urandom of=packed.exe bs=1 seek=$OFF count=$SZ conv=notrunc status=none
./meow packed.exe
The clean .text measured 5.744 (green/high); after randomization the same section jumps to 7.997, the bar goes full red, and the verdict flips to likely packed/encrypted:
That is the entire detector - a histogram, one logarithm sum, and two thresholds from a 2007 paper.
summary
| component | what it does |
|---|---|
entropy() (12 lines) |
shannon \(H\) of a byte buffer, 0..8 bits/byte |
analyze_pe() |
walk DOS/NT headers -> section table by exact offsets |
analyze_elf() |
walk ELF64 section headers, names from .shstrtab |
print_section() |
colored ascii bar + green/yellow/red thresholds |
| verdict | max section \(H\) vs the Lyda & Hamrock ~7.0 line |
A few honest limitations, because entropy is a blunt instrument. It flags unusual bytes, so legitimately compressed resources - icons, embedded installers, zipped payloads inside .rsrc - will also look “packed”. An attacker can also game the average by padding a section with low-entropy filler around a small encrypted blob, which is why per-section and even sliding-window analysis beats a single file-level number. And entropy cannot tell encrypted from merely compressed - both are near-uniform. As always, one signal is never a verdict: combine it with section context, the import table, and known-packer YARA rules.
In the next part we will fix the biggest of these gaps by adding a chi-square uniformity test on top of entropy - it separates encrypted data (which passes a randomness test) from compressed data (which usually does not), still in pure C and still in one tiny file.
I hope this post with practical examples is useful for malware researchers, reverse engineers and everyone interested in blue team skills like threat hunting.
Malware analysis part 6: Shannon entropy
Malware analysis part 10: Practical PE parsing
Malware analysis part 11: mini-GPT for binary analysis
C. E. Shannon - A Mathematical Theory of Communication (1948)
R. Lyda, J. Hamrock - Using Entropy Analysis to Find Encrypted and Packed Malware (IEEE S&P, 2007)
Microsoft PE Format specification
source code in github
This is a practical case for educational purposes only.
Thanks for your time happy hacking and good bye!
PS. All drawings and screenshots are mine
