Never Ending Security

It starts all here

Tag Archives: CTF

CTF Field Guide


CTF Field Guide

“Knowing is not enough; we must apply. Willing is not enough; we must do.” – Johann Wolfgang von Goethe

Welcome!

We’re glad you’re here. We need more people like you.

If you’re going to make a living in defense, you have to think like the offense.

So, learn to win at Capture The Flag (CTF). These competitions distill major disciplines of professional computer security work into short, objectively measurable exercises. The focus areas that CTF competitions tend to measure are vulnerability discovery, exploit creation, toolkit creation, and operational tradecraft.

Whether you want to succeed at CTF, or as a computer security professional, you’ll need to become an expert in at least one of these disciplines. Ideally in all of them.

That’s why we wrote this book.

In these chapters, you’ll find everything you need to win your next CTF competition:

  • Walkthroughs and details on past CTF challenges
  • Guidance to help you design and create your own toolkits
  • Case studies of attacker behavior, both in the real world and in past CTF competitions

To make your lives easier, we’ve supplemented each lesson with the Internet’s best supporting reference materials. These come from some of the best minds in the computer security field. Looking ahead, we hope you’ll collaborate to keep this book evolving with the industry.

We’ve tried to structure this so you can learn as quickly as you want, but if you have questions along the way, contact us. We’ll direct your question to the most relevant expert. If there’s enough demand, we may even schedule an online lecture.

Now, to work.

-The Trail of Bits Team

Why CTF?

Computer security represents a challenge to education due to its interdisciplinary nature. Topics in computer security are drawn from areas ranging from theoretical aspects of computer science to applied aspects of information technology management. This makes it difficult to encapsulate the spirit of what constitutes a computer security professional.

One approximation for this measure has emerged: the ‘capture the flag’ competition. Attack-oriented CTF competitions try to distill the essence of many aspects of professional computer security work into a single short exercise that is objectively measurable. The focus areas that CTF competitions tend to measure are vulnerability discovery, exploit creation, toolkit creation, and operational tradecraft.

A modern computer security professional should be an expert in at least one of these areas and ideally in all of them. Success in CTF competitions demands that participants be an expert in at least one and ideally all of these areas. Therefore, preparing for and competing in CTF represents a way to efficiently merge discrete disciplines in computer science into a focus on computer security.

Find a CTF

If you ever wanted to start running, you were probably encouraged to sign up to a 5k to keep focused on a goal. The same principle applies here: pick a CTF in the near future that you want to compete in and come up with a practice schedule. Here are some CTFs that we can recommend:

Visit CTF Time and the CapCTF calendar for a more complete list of CTFs occuring every week of the year.

How is a Wargame different?

Wargames are similar to a CTF but are always ongoing. Typically, they are organized into levels that get progressively harder as you solve more of them. Wargames are an excellent way to practice for CTF! Here are some of our favorites:

What about CCDC?

There are some defense-only competitions that disguise themselves as CTF competitions, mainly the Collegiate Cyber Defense Challenge (CCDC) and its regional variations, and our opinion is that you should avoid them. They are unrealistic exercises in frustration and will teach you little about security or anything else. They are incredibly fun to play as a Red Team though!

Career Cheatsheet

[Editor’s note: this is an older article written for pentest.cryptocity.net and that we are in the process of updating.]

These are my views on information security careers based on the experience I’ve had and your mileage may vary. The information below will be most appropriate if you live in New York City, you’re interested in application security, pentesting, or reversing, and you are early on in your career in information security.

  1. Employers
  2. Roles
  3. Learn from a Book
  4. Learn from a Course
  5. University
  6. University
  7. Communication
  8. Meet People
  9. Conferences
  10. Certifications
  11. Links
  12. Friends of the Guide

Employers

As far as I can tell, there are five major employers in the infosec industry (not counting academia).

  • The Government
  • Non-Tech Fortune 500s (mostly finance)
  • Big Tech Vendors (mostly West coast)
  • Big Consulting (mostly non-technical)
  • Small Consulting (mostly awesome)

The industry you work in will determine the major problems you have to solve. For example, the emphasis in finance is to reduce risk at the lowest cost to the business (opportunities for large-scale automation). On the other hand, consulting often means selling people on the idea that X is actually a vulnerability and researching to find new ones.

Roles

I primarily split up infosec jobs into internal network security, product security, and consulting. I further break down these classes of jobs into the following roles:

  • Application Security (code audits/app assessments)
  • Attacker (offensive)
  • Compliance
  • Forensics
  • Incident Handler
  • Manager
  • Network Security Engineer
  • Penetration Tester
  • Policy
  • Researcher
  • Reverse Engineer
  • Security Architect

The roles above each require a different, highly specialized body of knowledge. This website is a great resource for application security and penetration testing, but you should find other resources if you are interested in a different role.

Learn from a Book

Fortunately, there are dozens of good books written about each topic inside information security. Dino Dai Zovi and Tom Ptacek both have excellent reading lists. We recommend looking at:

If you’re not sure what you’re looking for, then you should browse the selection offered byO’Reilly. They are probably the most consistent and high-quality book publisher in this industry.

Don’t forget that reading the book alone won’t give you any additional skills beyond the conversational. You need to practice or create something based on what you read to really gain value and understanding from it.

Learn from a Course

If you’re looking for something more hands-on and directed, there are lots of university courses about information security available online. I listed some of the best ones that have course materials available below (ordered by institution name). The RPI course is the most similar to this one and Hovav gets points for the best academic reading list, but every course on this list is fantastic.

[Ed. note: Table to be added/updated later.]

University

The easiest shortcut to finding a university with a dedicated security program is to look through the NSA Centers of Academic Excellence (NSA-COE) institution list. This certification has become watered down as more universities have obtained it and it might help to focus your search on those that have obtained the newer COE-CO certification. Remember, certifications are only a guideline. You should look into the actual programs at each university instead of basing your decision on a certification alone.

Once in university, take classes that force you to write code in large volumes to solve hard problems. IMHO the courses that focus on mainly theoretical or simulated problems provide limited value. Ask upper level students for recommendations if you can’t identify the CS courses with programming from the CS courses done entirely on paper. The other way to frame this is to go to school for software development rather than computer science.

Capture the Flag

If you want to acquire and maintain technical skills and you want to do it fast, then you should play in a CTF or jump into a wargame. The one thing to note is that many of these challenges attach themselves to conferences (of all sizes), and by playing in them you will likely miss the entire rest of the conference. Try not to over do it, since conferences are useful in their own way (see the rest of the career guide).

There are some defense-only competitions that disguise themselves as normal CTF competitions, mainly the Collegiate Cyber Defense Challenge (CCDC) and its regional variations, and my opinion is that you should avoid them. They are exercises in system administration and frustration and will teach you little about security or anything else. They are incredibly fun to play as a Red Team though.

Communication

In any role, the majority of your time will be spent communicating with others, primarily through email and meetings and less by phone and IM. The role/employer you have will determine whether you speak more with internal infosec teams, non-security technologists, or business users. For example, expect to communicate more with external technologists if you do network security for a financial firm.

Tips for communicating well in a large organization:

  • Learn to write clear, concise, and professional email.
  • Learn to get things done and stay organized. Do not drop the ball.
  • Learn the business that your company or client is in. If you can speak in terms of the business, your arguments a) to not do things b) to fix things and c) to do things that involve time and money will be much more persuasive.
  • Learn how your company or client works, ie. key individuals, processes, or other motivators that factor into what gets things done.

If you are still attending a university, as with CS courses, take humanities courses that force you to write.

Meet People

Find and go to your local CitySec, an informal meetup without presentations that occurs once monthly in most cities. At Trail of Bits, we attend our local NYSEC.

ISSA and ISC2 focus on policy, compliance and other issues that will be of uncertain use for a new student in this field. Similarly, InfraGard mainly focuses on non-technical law enforcement-related issues. OWASP is one of the industry’s worst examples of vendor capture and is less about technology and more about sales.

Conferences

If you’ve never been to an infosec conference before, use the google calendar below to find a low-cost local one and go. There have been students of mine who think that attending a conference will be some kind of test and put off going to one for as long as possible. I promise I won’t pop out of the bushes with a final exam and publish your scores afterward.

If you go to a conference, don’t obsess over attending a talk during every time slot. The talks are just bait to lure all the smart hackers to one location for a weekend: you should meet the other attendees! If a particular talk was interesting and useful then you can and should talk to the speaker. This post by Shawn Moyer at the Defcon Speaker’s Corner has more on this subject.

If you’re working somewhere and are having trouble justifying conference attendance to your company, the Infosec Leaders blog has some helpful advice.

Certifications

This industry requires specialized knowledge and skills and studying for a certification exam will not help you gain them. In fact, in many cases, it can be harmful because the time you spend studying for a test will distract you from doing anything else in this guide.

That said, there are inexpensive and vendor-neutral certifications that you can reasonably obtain with your current level of experience to help set apart your resume, like the Network+and Security+ or even a NOP, but I would worry about certifications the least in your job search or professional development.

In general, the two best reasons to get certifications are:

  • If you are being paid to get certified, through paid training and exams or sometimes through an automatic pay raise after you get the certification (common in the government).
  • If your company or your client is forcing you to get certified. This is usually to help with a sales pitch, ie. “You should hire us because all of our staff are XYZ certified!”

In general, it is far more productive to spend time playing in a CTF, then using your final standing as proof that you’re capable.

Vulnerability Discovery

Auditing Source Code

This module is about getting familiar with vulnerabilities that manifest in applications that compile to native code. An accurate and complete understanding of an application written in a compiled language cannot be achieved without learning about how the compiler transforms source to machine code and how processors execute that code. An easy way to gain experience with these transforms is by reverse engineering compiled variants of your own code or of projects with source code available. At the end of this module you will be able to identify vulnerabilities in compiled languages like C and C++.

Vulnerabilities are commonly identified in large software packages due to their use of third-party software libraries. Common examples include libraries like libxml, libpng, libpoppler, and libfreetype that parse complicated file formats and protocols. Each of these libraries have historically been prone to software flaws that make the applications using them vulnerable to attack. It doesn’t help that most software packages fail to update these libraries when new versions come out, making it significant easier to find known vulnerabilities to apply in these cases.

Lecture

Workshop

In order to practice your skills, we recommend going through the process of identifying as many vulnerabilities as you can in an intentionally vulnerable toy application and then moving on to a real application and doing the same.

The Newspaper application is a small server written in C that allows authenticated users to read and write articles to a remote file system. Newspaper is written in such a way that it is vulnerable to many different attacks. You should be capable of identifying at least 10 bugs or potential vulnerabilities in this code.

Wireshark, however, is an industry standard network protocol analyzer that has been under continuous development since 1998. Vulnerabilities in this code base are much fewer and far between than in the Newspaper app however many still exist. Take a look at the wireshark security page, find the name of a protocol dissector and see if you can independently discover the vulnerability without looking at the details. Dissectors are located in /epan/dissectors/ folder.

Tools

When auditing, it is helpful to use a tool design to profile and navigate the codebase. Below are two tools, Source Navigator and Understand, designed to help analysts get familiar with code quickly by collecting and displaying information about data relationships, API usage, design patterns and control flow. An example of a useful diffing tool is also listed below. One example of a free, open source code auditing tool is the Clang Static Analyzer, which can help you track down programming errors in common APIs and vulnerable programming patterns.

Resources

Make sure you’re intimately familiar with the internals of the languages you target for analysis. Vulnerabilities are found by auditors who are more familiar with the language and the codebase than the programmers that originally developed it. In some cases, this level of understanding can be achieved simply by paying attaching to optional compiler warnings or through the use of third-party analysis tools that help track down common programming errors. Computer security is tantamount to computer mastery. Without a rigorous understanding of your targets you can never hope to defeat them.

Auditing Binaries

You’ve made it all the way down to the native layer, this is what software is after you pull off all the covers. The flavor of native code we’re going to focus on today is 32-bit Intel x86. Intel processors have been a powerful force in personal computing since the 80’s and currently predominate desktop and server market. Understanding this instruction set will give you some insight into how the programs you use every day operate as well as provide a reference point for when you encounter other instruction sets like ARM, MIPS, PowerPC and SPARC.

This module is about becoming familiar with the native layer and developing strategies for understanding, analyzing and interpreting native code. By the end of this module you should be capable of performing a “reverse compilation” — going from assembly fragments to statements in higher level languages — and, in the process, deriving meaning and programmer intent.

Lecture

Learning x86 can appear daunting at first and requires some dedicated study to master. We recommend reading Chapter 3 of “Computer Systems: A Programmer’s Perspective” to learn how C programs get compiled into machine code. Once you you have some basic, working knowledge of this process then keep a handy reference guide around like the x86 Assembly Guide from the University of Virginia. We’ve found this video series from Quinn Liu to be a quick and painless introduction too.

Workshop

The following programs are both “binary bombs.” Reverse engineer the following linux programs to identify the series of inputs that will “defuse” the bomb. Each successive level of the bomb focuses on a different aspect of native code. For example, in the lab from CMU you will encounter different data structures (linked lists, trees) as well as how control flow structures (switches, loops) manifest in native code. While reversing these programs you may find it useful to keep track of your progress by transforming what you see into C or another high level language.

You should aim to solve at least eight stages between the two labs. The CMU bomb lab has a secret phase and the RPI bomb lab has a phase that involves memory corruption, can you find and solve them?

Tools

The two essential tools for working with native code are the debugger and the disassembler. We recommend you become familiar with the industry standard disassembler: IDA Pro. IDA will divide code into discrete chunks corresponding to the functions defined in the program’s source code. Each function is then further subdivided into “basic blocks” defined by instructions that alter control flow. This makes it easy to identify loops, conditionals, and other control flow constructs at a glance.

Debuggers allow you to interact with and examine the state of running code by setting breakpoints and examining memory and register contents. You may find this useful as a sanity check if you are not seeing the results you expect your input to produce but be alert, some programs employ anti-debugger techniques and can modify program behavior in the presence of a debugger. The GNU Debugger (gdb) is the standard debugger for most linux systems. gdb can be acquired through the package manager in your chosen linux distribution.

Resources

Many good resources exist for learning x86 assembly and the various tricks employed in capture the flag exercises. In addition to the resources above, the x86 Wikibook and the AMD instruction set manual are more complete reference guides you may want to refer to (we find the AMD manual can be less daunting than the corresponding manual from Intel).

Some of the tools used for reverse engineering can be as complicated as assembly language itself. Cheatsheets that list out common commands and use cases can be helpful.

Finally, many capture the flag challenges will make use of anti-debugging and anti-disassembly techniques to hide or obfuscate the goal. Several of these techniques are employed by the bomb labs but you may want a more complete reference.

Auditing Web Applications

Welcome to the web hacking module. This module is about getting familiar with vulnerabilities commonly found in web applications. At the end of this module you will be able to identify common vulnerabilities in web based applications using a variety of testing methodologies and source level auditing. The lecture material will give you all the tools you need to successfully audit the workshop material.

Lecture

Workshop

In order to practice your skills, we recommend going through the process of finding and exploiting vulnerabilities in the Damn Vulnerable Web App (DVWA) and the Siberia Exploit Kit. DVWA is a collection of vulnerable test cases implemented in PHP and serves as an easy introduction to the many things that can go wrong in web applications. The Siberia Exploit Kit is a “crimeware pack” used by criminals to perform massive compromises. It includes a package of browser exploits and a command and control panel intended to manage compromised hosts. Siberia contains several pre- and post-authentication vulnerabilities that allow an attacker to gain administrative access to the panel, then take over the server on which it is hosted.

Download and run the OWASP Broken Web Apps virtual machine in VMware to start this workshop. BWA includes many web applications many for security testing, including DVWA. Once you have mastered DVWA, feel free to move on to other vulnerable web applications! Try auditing Siberia’s source code to find the vulnerabilities, paying attention to sources of input in PHP.

Tools

Burp Suite is a local HTTP proxy intended for security testing. Burp Suite is made for web penetration testers and simplifies many common tasks in a point-and-click GUI. The features available in the free version are more than enough to complete this and many other web security challenges.

Resources

Many simple testing methods and common web application flaws are available in the walkthrough. Ensure that you understand the fundementals of HTTP, HTML, and PHP to do well on this section.

Exploit Creation

Exploiting Binaries 1

Binary exploitation is the process of subverting a compiled application such that it violates some trust boundary in a way that is advantageous to you, the attacker. In this module we are going to focus on memory corruption. By abusing vulnerabilities that corrupt memory in software we can often rewrite critical application state information in a way that allows us to elevate privileges inside the context of a particular application (like a remote desktop server) or perform arbitrary computation by hijacking control flow and running code of our choosing.

If you’re trying to find bugs in compiled C programs, it’s important to know what you’re looking for. Start with identifying where the data you send to the program is used. If your data is stored in a buffer, take note of the sizes of them. Programming in C without errors is very difficult and the CERT C Coding Standard catalogues many of the ways that errors can come about. Paying attention to commonly misused APIs can be a quick path to success.

Once a vulnerability is identified it should be used to compromise the integrity of the program, however, there are a variety of ways to achieve this goal. For programs such as web servers, obtaining the information from another user may be the end goal. In others, changing your permissions may be helpful, for example changing the permissions of a local user to the administrator.

Lecture

The first lecture, Memory Corruption 101, provides background and step-by-step explanation of exploiting an overflow on Windows. The second lecture, Memory Corruption 102, covers more advanced topics, including web browser exploitation. Both of these lectures use Windows-specific examples but the techniques and process are applicable across operating systems that use the x86 instruction set. Remember that the names of functions and sometimes the calling conventions will be different when you are working with UNIX/Linux binaries.

Tools

We recommend using GDB to debug the challenges in this module since all of them are compiled for 32-bit Linux, however, GDB is intended for debugging source code, not stripped binaries without symbols and debugging information. Projects such as gdbinit, peda, andvoltron are attempts at making gdb more useful for debugging when source code is not available. We recommend making a .gdbinit file in your home directory with at least the following commands:

    set disassembly-flavor intel
    set follow-fork-mode child

Workshop

In order to run these challenges, you’ll need to setup an Ubuntu 14.04 (32-bit) virtual machine. We recommend using VMware Player, since it’s free and well supported. When you have it running, open a terminal and install socat with command sudo apt-get install socat.

There are three challenges in this workshop, each contained within this folder when you clone this repository in git. The ultimate goal of each challenge is to manipulate the executable into reading the flag to you. For each challenge, try to translate the disassembly into C code. After making an attempt, you can verify your guess by checking the actual C source provided. Then, try to exploit it to read you the flag.

Challenge: Easy

Make sure the flag is in the same directory as the easy program. Once you execute easy it will listen for instructions on port 12346.

Challenge: Social

Similar to easy, make sure the flag and host.sh are in the same directory as social program. Once you execute social it will listen for instructions on port 12347.

Resources

Exploiting Binaries 2

In this module, we continue to examine the ways that native applications can be exploited and focus on using return-oriented programming (ROP) to achieve that goal. ROP is the process of stitching together existing executable fragments of code ending in a return instruction. By creating chains of addresses of these ‘gadgets’ one can write new programs without introducing any new code.

Keep in mind that you will need to be flexible in identifying methods to exploit programs. Sometimes it’s necessary to abuse a vulnerability multiple times in the course of an exploit. At times, you may only want to use a ROP bridge to make your shellcode executable and, at others, you may want to use a payload written entirely in ROP. Occasionally, the layout of memory makes unorthodox methods of exploitation favorable. For example, have you considered manufacturing an uncontrolled format string vulnerability using ROP?

Lecture

The lectures this week will discuss return oriented programming (ROP) and code reuse to bypass protections that disallow the execution of attacker-provided data. These lectures go into much greater detail on exploitation and build upon some of what was discussed last week.

Workshop

Similar to the previous lesson, there are two executable files located in this folder when you clone the repository. Exploits for each of these programs will require the use of return-oriented programming to read the flags. This week, there is no access to source code provided. You will need to reverse engineer the binaries to discovery the vulnerabilities and the techniques required to exploit them. Use the same Ubuntu 14.04 (32-bit) virtual machine as the previous lesson.

Run the bc program. It will listen on port 12345.

Challenge: space

Run host.sh in the same directory as the space program. It will listen on port 12348.

Challenge: rop_mixer

Run host.sh in the same directory as the rop_mixer program. It will listen on port 12349.

Tools

Refer to the tools from last week. If you haven’t already, you should become familiar with thebinutils suite for *NIX. Tools like readelf, strings, objdump, objcopy, and nm are frequently helpful. Use your package manager and the manpages to install and read about their use.

Several tools exist to aid specifically in the creation of exploits that reuse code. They are more specialized than a standard disassembler because they look for executable snippets of code suitable as ROP gadgets (between instructions, in .rodata, etc).

Resources

Web Exploitation

This module follows up on the previous auditing web applications module. In this module we will focus on exploiting those vulnerabilities. By the end of this module you should be comfortable identifying and exploiting the OWASP Top 10.

Lecture

We covered the basics in the previous section on web security, so now we can dive into some more capable tools to achieve greater effects in this module. Learn to master Burp Suite and the Chrome Developer tools to gain a greater understanding of the applications you interact with. BeEF is an example of an XSS proxy and it will pay off to look through its source code and learn how it works.

Workshop

You have been tasked with auditing Gruyere, a small, cheesy web application. Gruyere is available through and hosted by Google. It includes exercises for exploiting many classes of web-specific vulnerabilities including XSS, SQL injection, CSRF, directory traversal and more. For each challenge you can find hints, exploits and methods to patch the vulnerable code.

References

Tools

SQL Map and BeEF are powerful tools and very fun to play around with but ultimately not needed for the exercise. If you insist on playing with BeEF on, then please try not to hook other users auditing the application.

Forensics

Toolkit Creation

Toolkit Prep

Welcome to the module on toolkit creation. A toolkit is a set of utilities that enable you and your team to achieve operational goals in the most efficient manner possible. Your toolkit is a force multiplier that will enable you to minimize the time you spend developing exploits during the game and maximize the return on your development time.

A good toolkit is well rounded and easy to use. You should incorporate software that allows members of your team to communicate effectively, work collaboratively, automate common tasks and provide situational awareness of the game as it plays out.

Lecture

Workshop

Create three lists. Populate the first list with the functionality your ideal toolkit would provide. Populate the second list with software that can provide that functionality. Use the third list to rank in order of importance functionality that is inadequately supported by the software from list two. Begin developing software that fills in the gaps of your ideal toolkit.

Some functionality you should not neglect:

  • Management of exploitation, key aggregation and submission.
  • Stealthy and secure payloads or persistence methods.
  • Secure communication and collaboration.
  • Network/Host situational awareness.

Resources

Operational Tradecraft

Studies in Tradecraft

Operational tradecraft is generally cultivated with a specific goal in mind. While playing competitive wargames you will most likely be focused on evading detection and not putting elements of your toolkit (infrastructure, exploits) at risk of inadvertent exposure.

Lecture

Workshop

Evaluate the operational tradecraft displayed during the following campaigns. Each design decision employed in these tools and campaigns has an operational philosophy behind it.

Some things to think about while evaluating tradecraft:

  • Why did the actor chose to perform/implement X action/capability?
  • Were any mistakes made? Was a decision flawed or shortsighted in some way?
  • Was an action/capability anomalus? Does it fit with the rest of the operational philosophy?
  • What was the actor most interested in protecting? (ex: Tools, Identities, Employers etc.)
  • What can be learned from each campaign from an attackers standpoint? Defenders standpoint?

Campaigns

Resources

These are few public examples, groups, or organizations that discuss their own tradecraft. The AMA’s below provide a rare glimpse into how extraordinarily talented groups operate.

Contributors

This book was built on a lot of hard work, most of which happened elsewhere. Without permission to republish from these folks, this book would still be in development. They have our gratitude and admiration.

So, reader, when you’ve completed a few CTFs, and you’re ready for more, reach out to this list. They like to support ambition, and they just might know someone who needs some talent.

If you’re interested in taking a course like this one for credit, check out NYU Poly. They offer concentrations in cybersecurity and we collaborate with them frequently through their Hacker in Residence program.

Contributing

If you want to make a contribution, simply commit your new markdown to the master branch and we’ll take it from there. Gitbook has a fantastic editor available to help preview your changes. We’re always looking for new or refined content, so please send us your suggestions!

Gitbook Usage

The CTF Field Guide is built with Gitbook, a command line tool for building books with Git and Markdown. You can use Gitbook to output the CTF Field Guide as a PDF, an eBook or a single, printable HTML page. Make sure you have NodeJS and npm on your operating system, then install Gitbook and a few of its plugins:

npm install gitbook gitbook-plugin-ga gitbook-pdf ebook-convert -g

With Gitbook installed, you can run any of these commands from within the book directory:

  • Generate an interactive, static website: gitbook build ./myrepo
  • Generate a single page website: gitbook build ./myrepo -f page.
  • Generate a PDF: gitbook pdf ./myrepo. Requires gitbook-pdf.
  • Generate an eBook: gitbook ebook ./myrepo. Requires ebook-convert.

More information and the complete book can be found at:

https://trailofbits.github.io/ctf/

CTFd – CTFs as you need them


logo

Capture The Flags (CTFs) are how the those of us in the computer security industry test both ourselves and each other. This year for CSAW CTF I completely rewrote the website behind the CTF to make it easier for myself and future organizers to administrate.

CTFd, as I’ve named this software it, has plenty of uses outside the competitive scene and we’ve used it to teach our Hack Night program and as recruitment for our CTF team.

CTFd has everything you need to run a large scale CTF or a small classroom exercise. CTFs are one of the fastest growing educational mediums and their versatility in both testing and teaching is unmistakable.

Prioritizing ease of deployment, administration, and use, CTFd takes less than 5 minutes to get onto a network and there are very few scenarios in which you have to leave your web browser to manage it.

Today, I’m happy to both open source CTFd and provide a hosted solution available to everyone.

CTFd development is ongoing, and there are so many more features to add, but I hope that you can make as much use of CTFd as we have.

Wiki-like CTF write-ups repositories, maintained by the community. 2013, 2014 and 2015


CTF write-ups 2013-2014-2015

There are some problems with CTF write-ups in general:

  • they’re scattered across the interwebs
  • they don’t usually include the original files needed to solve the challenge
  • some of them are incomplete or skip ‘obvious’ parts of the explanation, and are therefore not as helpful for newcomers
  • often they disappear when the owner forgets to renew their domain or shuts down their blog

This repository aims to solve those problems.

It’s a collection of CTF source files and write-ups that anyone can contribute to. Did you just publish a CTF write-up? Let us know, and we’ll add a link to your post — or just add the link yourself and submit a pull request. Spot an issue with a solution? Correct it, and send a pull request.

Archive

  • Write-ups for CTFs that occurred in 2013https://github.com/ctfs/write-ups-2013
  • backdoorctf-2013
    csaw-quals-2013
    hack-lu-ctf-2013
  • Write-ups for CTFs that occurred in 2014https://github.com/ctfs/write-ups-2014
  • 31c3-ctf-2014
    9447-ctf-2014
    asis-ctf-finals-2014
    asis-ctf-quals-2014
    boston-key-party-2014
    codegate-preliminary-2014
    confidence-ds-ctf-teaser
    csaw-ctf-2014
    d-ctf-2014
    def-con-ctf-qualifier-2014
    defkthon-ctf
    ectf-2014
    ghost-in-the-shellcode-2014
    ghost-in-the-shellcode-2015-teaser
    gpn-ctf-2014
    hack-in-the-box-amsterdam-2014
    hack-lu-ctf-2014
    hack-you-2014
    hitcon-ctf-2014
    ncn-ctf-2014
    ncn-ctf-quals-2014
    notsosecure-ctf-2014
    nuit-du-hack-ctf-qualifications
    olympic-ctf-2014
    phdays-iv-finals
    phdays-iv-quals
    pico-ctf-2014
    plaid-ctf-2014
    pwnium-ctf-2014
    qiwi-ctf-2014
    ructf-2014-quals
    ructfe-2014
    seccon-ctf-2014
    secuinside-ctf-prequal-2014
    stripe-ctf3
    su-ctf-quals-2014
    tinyctf-2014
    volga-quals-2014
  • Write-ups for CTFs that occurred in 2015https://github.com/ctfs/write-ups-2015
  • 0ctf-2015
    backdoor-ctf-2015
    bctf-2015
    boston-key-party-2015
    break-in-ctf-2015
    bsides-vancouver-ctf-2015
    codegate-ctf-2015
    codegate-ctf-junior-2015
    cyber-security-challenge-2015
    ghost-in-the-shellcode-2015
    ibteam-blackvalentine-ctf-2015
    insomni-hack-ctf-2015
    insomni-hack-ctf-teaser-2015
    nuit-du-hack-ctf-quals-2015
    nullcon-hackim-2015
    opentoall-ctf-2015
    pragyan-ctf-2015
    sCTF-2015
    securinets-ctf-2015
    th3jackers-ctf-2015
    uiuctf-2015

A collection of tools used to maintain and create CTF writeup folders:
https://github.com/ctfs/write-ups-tools

Generate a CTF directory/skeleton

Use this tool to generate a CTF skeleton.

This is how I usually maintain a new CTF directory/skeleton

  • Create an empty directory for the CTF, ending with the current year, e.g. example-ctf-2015/
  • Create an empty directory in this new CTF directory for each task category, e.g. mkdir crypto web misc trivia
  • Create an empty for each task in the according category folder, e.g. mkdir crypto/{rsalot, rsanne}
  • Download all CTF files during the CTF and save the description, points, original task name, solves and task category for each file in a file named info, e.g. crypto/rsalot/info and crypto/rsanne/info
  • Generate a README.md for each info file in the CTF directory using the genctf.py tool, e.g. python genctf.py example-ctf-2015/ info 'Example CTF'
  • Remove all info files (make a backup of your CTF directory just in case) using find example-ctf-2015 -name info -delete
  • Tell git to ignore all files that are bigger than 10MBytes with cd example-ctf-2015; find . -size +10M >> .gitignore
  • Edit each README.md to fill in missing information (e.g. Authors, references and solves)
  • Move the CTF directory to the write-ups-$YEAR repo, making sure that it not yet exist

A general collection of information, tools, and tips regarding CTFs and similar security competitions:
http://ctfs.github.io/resources

CTF Resources

(https://github.com/ctfs/resources)

This repository aims to be an archive of information, tools, and references regarding CTF competitions.

CTFs, especially for beginners, can be very daunting and almost impossible to approach. With some general overviews of common CTF topics and more in-depth research and explanation in specific technologies both beginners and veterans can learn, contribute, and collaborate to expand their knowledge.

Quick Start

  1. First time? READ THIS! and then the section below. Once you understand the basics, use the resources in the topics directory to try to solve challenges on websites like OverTheWire orCanYouHack.it.
  2. Beginner? Use the guides found in the topics directory to try to find out what type of challenges you are presented with and participate in some of the CTFs on ctftime.
  3. Intermediate? Navigate straight to the topic you are interested in to find extra online resources to help you solve more complex challenges.
  4. Master? Help improve this repository! Have a new type of vulnerability you want to explain? Write about it and how to use it! Have a new tool people can use? add it to the tools directory!

What are CTFs?

CTFs are computer security/hacking competitions which generally consist of participants breaking, investigating, reverse engineering and doing anything they can to reach the end goal, a “flag” which is usually found as a string of text.

DEF CON hosts what is the most widely known and first major CTF, occurring annually at the hacking conference in Las Vegas. Many different competitions have branched off since then, and numerous ones are available year round. One of the best places to see when CTFs are being scheduled isCTFTime, an active website with calendars and team rankings.

Example

A very simple type of CTF challenge consists of looking at the source code of websites or programs to find flags and/or hints. For example, can you find the flag hidden on this page?

Moving On

You may be able to solve some CTF challenges after looking through the documents in this repository and understanding the basics of the technologies and subjects covered, but you won’t be very proficient or successful for long. To be an adept CTF competitor you have to be able to combine many different strategies and tools to find the flag. Developing the ability to find flags quickly takes practice more than anything, and participating in numerous CTFs will allow you to expand your understanding and abilities, leading you to success. Spend some time on CTFTime working through CTFs to truly improve and learn.

Conclusion

Now that you know the basics of CTFs, you can visit ctftime and try out a CTF! Using your background knowledge and the information on this page you’ll be able to develop a solid basis in computer security.


Cryptography

Cryptography is the practice and study of techniques for secure communication in the presence of third parties. – Wikipedia

In the case of CTFs, the goal is usually to crack or clone cryptographic objects or algorithms to reach the flag.

Example

If you look around the folders in this page you should be able to find a suitable way to solve this simple cipher:

Hint: Julius Caesar's favorite cipher

kxn iye lbedec

Getting Started

To-Do

Sources/See More

Introduction to Cryptography


Caesar Cipher

The Caesar Cipher is a very simple and common encryption method which does not appear often in full-fledged CTFs but forms part of the basis of cryptography. It simply shifts a string of letters a certain number of positions up or down the alphabet.

Let’s say we want to encrypt the string hello world to give to our friend whose favorite number is 3. We will shift our string left 3.

Taking the first letter h in our string and going 3 places up the alphabet(as it is a left shift) gives us the letter e. We then start our new, encrypted string with the letter.

Doing so for the whole original string creates a jumbled mess of incomprehensible letters to anyone but the reader with the proper decryption shift:

Original: hello world

Final: ebiil tloia

To let our friend read this, we would send him the final string with the instructions right 3, and either by hand, with a website, or with a script, he would be able to extract our message.

Detecting

Caesar ciphers are usually presented in very low-point tasks, if at all, and can be easy to detect and check for. Strings containing incomprehensibly jumbled letters are possible Caesar ciphers and should be checked.

Solving

There are many approaches to cracking Caesar ciphers, but usually the best way to solve them is to write a script or run the string through a website which will print out all the possible shifts of a string. From those results the most comprehensible and logical solution can be chosen.

CTF Example

To-do

Sources/See More

Brute force caeser cipher cracker


Vigenère Cipher

The Vigenère Cipher is a method of encrypting alphabetic text by using a series of different Caesar ciphers based on the letters of a keyword – Wikipedia.

Please read the article on Caesar Ciphers if you haven’t already because the Vigenère Cipher is a direct derivative of the former. The Vigenère cipher takes a keyword and applies a certain caeser cipher to it multiple times according to the letters of a keyword.

To-Do Example

Detecting

Vigenère Ciphers appear to be identical to any other substitution cipher, but trying to solve it as Caesar Cipher will not work. Check for this type of cipher if the Caesar Cipher crack does not work.

Solving

To-Do

CTF Example

DEKTHON 2014 had a simple vigenère cipher with no hints and only a line of text:

ucoizsbtkxhtadcg

Solution can be found here.

Sources/See More

Online Vigenère cracker


MD5 Hashing

MD5 is a widely used cryptographic hash function producing a 128-bit (16-byte) hash value, typically expressed in text format as a 32 digit hexadecimal number –Wikipedia.

This system is commonly used to check the integrity of files (like downloads). The way MD5 hashes are created, any slight variation in a file creates a new hash that is completely different than the previous, making changes in files (e.g. corruption in download or tampering) very apparent.

Creating an MD5 hash is very simple, as there are multiple online tools like md5-creatorand even a command line tool md5sum which will quickly create a sum from input.

Detecting

MD5 hashes are very standard, as they are always 128 bits, or 32-character strings.

Solving

To-Do

Sources/See More

Easy MD5 cracker


RSA

To-do

Detecting

To-do

Solving

To-do

CTF Example

BackdoorCTF 2014 had an RSA challenge which simply provided a public key and encrypted text file.

The solution can be found here.

Sources/See More

CTF Write-up


Steganography

Steganography is the art or practice of concealing a message, image, or file within another message, image, or file. – Wikipedia

In the context of CTFs steganography usually involves finding the hints or flags that have been hidden with steganography. Most commonly a media file will be given as a task with no further instructions, and the participants have to be able to uncover the message that has been encoded in the media.

Example

Images are a very common medium for steganography, as they are easy to manipulate and simple to view and transport. Files in Images give a good introduction for beginner steganography.

Getting Started

A rudimentary knowledge of media filetypes (e.g. jpg, bmp, png for pictures and wav, mp3 for sound) is essential to steganography, as understanding in what ways files can be hidden and obscured is crucial. Also, understanding basic linux is important, as a multitude of tools are specifically for bash.

Sources/See More


Hiding a file in an image

One of the most common steganography tricks is to hide a file inside of an image. The file will open normally as an image but will also hold hidden files inside, commonly zip, text, and even other image files.

The reason this works is because when an image file is read it has starting and ending bytes dictating the size of the image. The image viewer that you use will use the information between these bytes to present an image to you, ignoring anything after the terminating byte.

For example, The terminating byte for a JPEG is FF D9 in hex, so using a hex viewer (xxdis good for linux, or something like HxD for windows) you can find out where the image finishes. These bytes are sometimes hard to find in a sea of numbers though, so looking at the dump of the hex (the text representing the hex bytes) can also help you find hidden .txt or .zip files.

Example

A very simple implementation of this strategy is used in the example.jpg file in this directory. If you save it to your computer and open it up with an image viewer, you should be presented with a simple jpg image.

Now lets try to find the flag. Open up the image in your favorite hex editor and start looking around for something odd (You may find the flag itself from the dump at this point, but for the sake of example try extracting it). Near the bottom of the file you should see the terminating byte of a jpg ffd9:

01e17a0: 685c 7fab 8eb4 5b32 61f1 c4ff d950 4b03 h\....[2a....PK.

Another important part of this line is the PK near the end. PK are the initials of Phil Katz, the inventor of the zip file, and indicate that a zip file starts at that point.

Using this information we can use another handy linux tool, dd). The dd command is very versatile and allows for the copying and converting of a multitude of files. In our case, we are going to be using it to extract the zip file.

We know where the location of the zip file is, but dd only takes decimal values, so we convert the hexadecimal location 0x01e17ad from hex to decimal to get 1972141.

Pluging this into dd:

dd if=example.jpg bs=1 skip=1972141 of=foo.zip

This takes in the image example.jpg, the ‘in file’ if, reads one block at a time, ‘block size’ bs, skips to block 1972141, skip, and writes it to the ‘out file’ zip we call foo.zip. When this completes you should have a zip file you can easily unzip to access the text file inside.

This is the long way of solving a simple steganography problem but shows how the strategy works. In the Solving section more concise and efficient methods are described.

Detecting

These challenges are usually presented as a simple picture with no other instructions, and it is up to the competitor to run it through a hex editor to find out if it involves steganography. If you are presented with an image and no instructions, your safest bet is that is has something hidden after the closing tags of the image.

Solving

Although it is possible and at times practical to solve these tasks using linux tools likedd, there are some tools that make it much easier. Binwalk is an immensely useful tool which automatically detects and extracts files hidden with steganography tools

CTF Example

Steganography of this type is usually not scored very highly but is decently widespread. BackdoorCTF 2014 created one which is generally straightforward, ctfexample.jpg, but involves multiple layers.

Sources/See More

XXD

HxD

DD

Binwalk


Hidden Text in Images

A simple steganography trick that is often used for watermarks instead of outright steganography is the act of hiding nearly invisible text in images. The text can be hidden by making it nearly invisible (turning down it’s opacity to below 5%) or using certain colors and filters on it. Although the text is undiscernable to the naked eye, it is still there, and there are a variety of tools which allow the text to be extracted.

Example

Using the tactics detailed below, can you find the flag in this image?

flag

Detecting

Detecting this type of steganography can be somewhat challenging, but once you know it is being used there are a multitude of tools you can use to find the flag. If you find that there are no other files hidden in the image (e.g. .zip files), you should try to find flags hidden with this method.

Solving

There are multiple ways to find flags hidden in this manner:

  • GIMP or Photoshop can be used to uncover the flag by using different filters and color ranges. This tutorial works remarkably well for finding hidden text.
  • Stegsolve is an immensly useful program for many steganography challenges, allowing you to go through dozens of color filters to try to uncover hidden text.
  • There are many scripts that have been written to substitute certain colors and make hidden the text legible, for example this Ruby script highlights colors passed to it in the image.

CTF Example

PlaidCTF 2014 had a steganography challenge recently with this image:

ctf-example

The write-up for this challenge can be found here

Sources/See More


Web

Web challenges in CTF competitions usually involve the use of HTTP (or similar protocols) and technologies involved in information transfer and display over the internet like PHP, CMS’s (e.g. Django), SQL, Javascript, and more. There are many tools used to access and interact with the web tasks, and choosing the right one is a major facet of the challenges. Although web browsers are the most common and well known way of interacting with the internet, tools like curl and nc allow for extra options and parameters to be passed and utilized.

Example

To-Do (need a website/server)

Getting Started

Command Line and the Web

If you are running linux and want extended functionality (like passing custom headers) in web challenges, bash (terminal) commands are your best bet. cURL is a simple but extensible command-line tool for transferring data using various protocols, and allows users to use HTTP to interact with servers, including POST and GET methods.

Example

To see curl at work, you can simply run curl 8.8.8.8 (Google), and the html of Google’s home page should appear.

There are many other options and flags that can be passed to curl, making it an extremely useful tool in CTFs

Sources/See More

HTTP

cURL


HTTP (Hypertext Transfer Protocol)


PHP

PHP is a server-side scripting language designed for web development.

Sources/See More

PHP


SQL Injections


Miscellaneous

Many challenges in CTFs will be completely random and unprecedented, requiring simply logic, knowledge, and patience to be solved. There is no sure-fire way to prepare for these, but as you complete more CTFs you will be able to recognize and hopefully have more clues on how to solve them.

Examples

In recent CTFs the sheer variety of miscellaneous tasks has been highly exemplified, for example:

Sources/See More

CTF Write-Ups

CTFTime