Investigate Cole

This was a red-teaming challenge I made back in June 2025, when I was just getting started with messing around with LLMS.

I designed it to explore the vulnerabilities of Large Language Models through prompt injection. Please bear with me if this sounds incredibly basic.

The LLM uses a system prompt on a scenario which involves Cole, a fictional law firm partner who has access to sensitive client data. Interns had to exploit the LLM’s behavior and bypass filters to uncover confidential information.

Beyond this challenge design, I also built the actual tool that the interns will use to interact with Cole. I wrote a custom JavaScript script using Node.js that:

• Used the Ollama API to run a local LLM model
• Created a CLI-based chat interface using node’s readline module
• Displayed real time output with a beautiful (in my opinion) spinner animation while the LLM is processing
• Used 3 rounds of obfuscation to make it difficult to reverse engineer.
  • First, I obfuscated the prompt
  • Then, I obfuscated the entire code
  • Lastly, I bundled all the packages together, and then obfuscated it once again
• Lastly, I used Node’s new Single Executable Applications feature to package everything up so Interns could run it locally.

What I Learned – Node.js App Development#

• API Integration: I learned how to integrate the ollama.chat() streaming API to send and receive real-time responses from a local LLM instance, enabling a smooth conversational interface.
• Initial Reasoning Behind Using Node.js: I initially chose Node.js under the assumption that it would be more resistant to reverse engineering. I initially assumed the compiled nature of the executable, along with students’ general unfamiliarity with JavaScript internals, would make the code harder to analyze.
• Obfuscation & Encoding: I later realized that JavaScript, especially when bundled into executables, is incredibly easy to reverse engineer. A simple strings command or opening the binary in a hex editor often reveals the entire source code in plaintext, which led me to apply multiple layers of obfuscation and encoding to significantly increase the difficulty of understanding or tampering with the code. While this was an effective hurdle for a CTF, it reinforced that client-side obfuscation is merely 'security by obscurity.' In a real production environment, the system prompt must be isolated on a secure backend server.
• Executable Packaging: I learned how to bundle Node.js packages into a single file using esbuild, and then package it into a standalone binary using Node's Single Executable Applications feature. Although convenient, I found that these executables embed the original source code directly—making obfuscation not just helpful, but absolutely necessary.

What I Learned - LLM & Security Concepts#

• Prompt Injection Tactics: Explored how attackers use prompt injection to bypass safety filters and coerce unintended outputs.
• System Prompt Crafting: I learned how to prompt an LLM, and what safeguards I should take to secure it and prevent any security vulnerabilities. For the sake of the CTF challenge, I intentionally skipped a few safeguards to make it slightly easier to solve.
• LLM Model Comparison:
  • I tested several open-source models, with the same prompt on all of them:
  • LLaMA 3.2 - Found this to be the most susceptible to prompt injection, often revealing restricted info or breaking character with minimal coaxing.
  • Phi-3 - This was much more resistant, harder to jailbreak, and I honestly wasn’t able to break this at all.
  • Mistral - This one showed intermediate levels of resistance, but repeated prompts again and again seemed to break it.