A walkthrough of the 'mistake' challenge

Chris Long

5 minute read


As my first post, I thought I would do a quick writeup of the Mistake challenge found on This challenge took me a bit longer than I expected, but the mistake does turn out to have interesting side effects.

The challenge is as follows:

We all make mistakes, let’s move on.

(don’t take this too seriously, no fancy hacking skill is required at all)

This task is based on real event

Thanks to dhmonkey

hint : operator priority

We’re presented with the following C program:

#include <stdio.h>
#include <fcntl.h>

#define PW_LEN 10
#define XORKEY 1

void xor(char* s, int len){
	int i;
	for(i=0; i<len; i++){
		s[i] ^= XORKEY;

int main(int argc, char* argv[]){

	int fd;
	if(fd=open("/home/mistake/password",O_RDONLY,0400) < 0){
		printf("can't open password %d\n", fd);
		return 0;

	printf("do not bruteforce...\n");

	char pw_buf[PW_LEN+1];
	int len;
	if(!(len=read(fd,pw_buf,PW_LEN) > 0)){
		printf("read error\n");
		return 0;

	char pw_buf2[PW_LEN+1];
	printf("input password : ");
	scanf("%10s", pw_buf2);

	// xor your input
	xor(pw_buf2, 10);

	if(!strncmp(pw_buf, pw_buf2, PW_LEN)){
		printf("Password OK\n");
		system("/bin/cat flag\n");
		printf("Wrong Password\n");

	return 0;

After looking over it a bit, we can break down what’s happening into a few steps:

  1. The program is getting the file descriptor for a file called password that we don’t have permission to read.
  2. It’s preventing us from bruteforcing it by adding a sleep statement
  3. It then reads PW_LEN bytes (10) from the file descriptor and stores them in the character array
  4. The program then initializes another character array and asks us to input a password.
  5. It stores 10 bytes from our input into pw_buf2 using scanf() and then calls xor() on it.
  6. We can see that the xor() function will xor each byte of our password with 1, as it’s defined in the macro.
  7. It then compares our xor’ed password buffer with the original password buffer that contains the contents of /home/mistake/password.
  8. If the contents match, we win!


The way I got started was by creating a copy of this program and adding debug statements throughout it. It also helps to comment out the sleep statement and create a local password file that the program can read from. The password should be 10 characters.

There’s also a clue early on, but I didn’t notice it until later. When you run the original program, it sleeps somewhere between 0 and 20 seconds. After that, it should immediately prompt us for input and display input password:. However, it doesn’t bring up that prompt until you hit enter. I totally overlooked this for awhile, but I did think it was kind of weird.

The two lines that stood out as weird to me were:

if(fd=open("/home/mistake/password",O_RDONLY,0400) < 0){

if(!(len=read(fd,pw_buf,PW_LEN) > 0)){

There was a lot to unpack from these two lines (for someone not familiar with C), so I started on the first one. I got hung up on the fact that no matter what I gave the program as input (via stdin or stdout), somehow len always ended up being 1 and fd was always 0. I was so convinced that the unchanging len was the problem that I was overlooking the root cause.

Instead of trying to figure out what exactly was wrong, I decided I would re-write those two concerning looking lines and figure out how the program would run if it didn’t contain a mistake! I rewrote those two lines to the following:

if(fd < 0){
printf("can't open password %d\n", fd);
return 0;
printf("fd value: %d\n", fd);
if(!(len > 0)){
printf("read error\n");
return 0;
printf("len value: %d\n", len);

Fixing the Mistake

I noticed 3 changes after “fixing” the code:

  1. fd was no longer always 0
  2. len was always 10 instead of always 1
  3. The program didn’t hang before displaying the input prompt

I had been assuming that open() returned 0 if the function was successful instead of realizing that it was returning a file descriptor. I believe the reason it’s incorrectly being set to 0 is because the open function open("/home/mistake/password",O_RDONLY,0400) < 0){ evaluates to 3, and 3 is not less than 0.

If you’re familiar with *NIX, you’ll recognize that 0 is the file descriptor for STDIN. The reason the program was hanging after the sleep statement was because it was trying to read from STDIN instead of reading from /home/mistake/password!

What does this mean for us? We control the contents of pw_buf (it’s what we pass via STDIN) and pw_buf2 (it’s what we pass at the input prompt).

We control both passwords now!

With that revelation under our belt, we just need to craft a 10 character password and figure out what that password becomes after each byte of it is xor’ed with 1. I decided to set a password of BBBBBBBBBB. We know from ASCII that B == 42, so we can calculate that 42 XOR 1 is 43, which is the letter C. Now we know if we supply a second password of CCCCCCCCCC, it will be XOR’ed with 1 again and the result will be BBBBBBBBBB.

To clarify, here’s what happens step by step:

mistake@ubuntu:~$ ./mistake
do not bruteforce...

The 10 B’s entered via STDIN will be stored in pw_buf because the mistake in the open() call line causes fd to be set to 0, causing the program to read 10 bytes from STDIN instead of the password file.

input password : CCCCCCCCCC

The input from this prompt will be stored in pw_buf2 and then the contents will be sent to the XOR function where each byte will be XOR’ed with 1. After the XOR function, pw_buf2 will contain the same contents as pw_buf. The program will do a strncmp() of pw_buf and pw_buf2, determine they are the same, and voila!

Password OK
Mommy, the operator priority always confuses me :(
comments powered by Disqus