introvoyz041/terminal-bench-2.0
terminal-bench-2.0: Harbor dataset on Hugging Face with 89 tasks. See what each task asks, how it is graded and what it runs in, then run an agent on it.
Tasks
- Your task is to implement an adaptive-rejection sampler as described in Gilks et al. (1992).
- You are provided with a dataset sample from a Bayesian Network in /app/bn sample 10k.csv.
- There's a /app/filter.py script that attempts to remove JavaScript from HTML files to prevent XSS attacks.
- pyknotid is a Python package for computational topology and knot theory.
- Build pMARS (http://www.koth.org/pmars/) from source with no X server.
- Build POV-Ray 2.2. Find and download the source archives, extract them to /app/povray-2.2 , then compile and install to…
- Install the original BVLC Caffe deep learning framework (version 1.0.0) and train a
- Create a Python function called async run tasks(tasks: list Callable , Awaitable None , max concurrent: int) - None where each task is an…
- The file chess board.png has an image of a chess board. It is currently white to move. Write the best move for white to play to…
- The file /app/sim.c is a logic-gate simulator: each line of the file /app/gates.txt should contain a line of the form
- You are given a COBOL program located at /app/src/program.cbl. This program reads input data from /app/src/INPUT.DAT and modifies one or…
- You're given an image of a pseudocode snippet at /app/code.png . Implement the intended logic of that snippet in any language and produce…
- Under /tmp/CompCert/, build the CompCert C verified compiler (version 3.13.1) from source,
- Configure a git server so that I can run on my computer
- Find a 1-hour meeting slot for Alice, Bob, and Carol during January 15-19, 2024 (business hours: 9 AM - 6 PM, Monday-Friday) that…
- Tell me how many deepseek tokens are there in the science domain of the ryanmarten/OpenThoughts-1k-sample dataset on huggingface.
- You need to create a file called "/app/solution.txt" with the word found in "secret file.txt" in the "secrets.7z" archive.
- I have a program at /app/ that crashes under RELEASE mode but not under DEBUG mode.
- I have a database in WAL (Write-Ahead Logging) mode in /app/.
- Finding a target probability distribution for LLM confidence metrics:
- The file titled sequences.fasta contains the following sequences:
- The file titled sequences.fasta contains the sequence for a circular input plasmid, and a desired output plasmid. Design primers so that…
- I have provided a file a.out that's a compiled C binary. Write me a program extract.js that, when run with node extract.js /app/a.out…
- Download this video of someone playing zork. https://www.youtube.com/watch?v=ZCbvyPbhRfA. Then transcribe the entire contents of the text…
- The file /app/feal.py implements a FEAL-like encryption function. Implement a chosen plaintext attack that recovers the value of key 5 .
- The file /app/feal.c implements a FEAL-like encryption function,
- Create a python file /app/filter.py that removes JavaScript from HTML files to prevent XSS attacks. You should preserve as much of the…
- You have a collection of mixed document files in the /app/documents/ directory containing both JPG and PDF files.
- You need to identify and fix the vulnerability in the code according to the Common Weakness Enumeration (CWE).
- I just made some changes to my personal site and checked out master, but now I can't find those changes. Please help me find them and…
- I have been making some changes to the OCaml garbage collector, improving the sweeping performance by run-length compressing the free…
- I have provided a file called text.gcode. This is a gcode file for a Prusa MK4s. I'm trying to print some text onto an existing object…
- A secret was accidentally committed to this repository in /app and then removed by rewriting history.
- Set up a Git server that hosts a project over SSH at git@localhost:/git/project.
- I have downloaded the gpt-2 weights stored as a TF .ckpt. Write me a dependency-free C file that samples from the model with arg-max…
- Implement the provided BaseTerminal interface. It provides a simple python interface to send keys to a headless terminal.
- Set up a local service to run inference with a Hugging Face transformer model.
- Run Windows 3.11 for Workgroups in a virtual machine using qemu. You image is in /app/isos/win311.img . This image is known to be…
- You need to build a simple KV store server using grpc that records number values for different string keys. Your server will use a Python…
- Transform /app/input.csv (1 million rows) to match /app/expected.csv exactly using keystroke-efficient Vim macros. Save your script as…
- Complete the implementation in find dominant eigenvalue and eigenvector in /app/eigen.py.
- In this task you will implement an LLM inference batching scheduler (shape‑aware)
- You are given multiple log files stored in /app/logs. Each log file name follows the pattern YYYY-MM-DD <source .log (e.g., 2025-08-10…
- Spin up a mailing list server for our reading group: reading-group@local.edu using postfix and mailman3 (both are installed already).
- I have provided /app/doomgeneric/, the source code to doom. I've also wrote a special doomgeneric img.c that I want you to use which will…
- I have provided /app/doomgeneric mips, a MIPS elf file, along with doomgeneric/, the corresponding source code. Please implement a MIPS…
- Sample from a hierarchical Bayesian model using R and Stan, and estimate the posterior means of the parameters.
- mkdir /app/repo, then initialize a git repo at /app/repo.
- I have a ReLU neural network that you can query by importing forward.py and calling forward(x) which returns a single floating point…
- Modernize Legacy Scientific Computing Stack
- I have a lot of Scandinavian texts that I need to encode. Please provide me with the name in organization/model name format (e.g…
- Given the query "terminal-bench" retrieve the document with the 5th highest cosine similarity among the texts at /app/data.txt, where each…
- Merge user data from three different sources with different formats and schemas.
- Set up an Nginx web server with advanced request logging and custom configurations. Your task is to:
- Your company needs a self-signed TLS certificate for an internal development server. Create a self-signed certificate using OpenSSL with…
- Ensure that the LaTeX document main.tex compiles successfully using the installed TeX distribution and pdflatex compiler with no "overfull…
- You need to perform a digital forensic recovery task. A system administrator has accidentally deleted an important file containing a…
- I've put an image at /app/image.ppm that I rendered programmatically. Write a c program image.c that I can run and compile and will…
- I've compiled a program at /app/mystery. Write a C program /app/mystery.c that performs an identical operation. You can do this by…
- Write me a single file in /app/polyglot/main.py.c which is a polyglot, that is, I can run
- Write me a single file in /app/polyglot/main.rs which is a polyglot, that is, I can run
- You have a Python baseline in portfolio baseline.py that calculates portfolio risk and return using nested loops. Complete the provided…
- I am planning an experiment where I'll be testing the stability of dihydrofolate reductase (DHFR) with FRET.
- Fix the incomplete proof of addition commutativity in the file plus comm.v. The file contains a partial proof that needs to be completed.
- Create a python package called vectorops, then build this package and set up a pypi server on port 8080 locally which also hosts this…
- Your task is to implement a command line tool that can be used to run inference on an MNIST model.
- - You are given a PyTorch state dictionary (/app/weights.pt) representing the weights of a Pytorch model, and a dataset (/app/dataset.pt)…
- Start the /app/alpine.iso image in qemu, and then start an ssh server on the alpine linux vm. When you're done, I should be able to run…
- Start the /app/alpine.iso image in qemu in such a way that I can connect to it via telnet 127.0.0.1 6665 . When I run telnet I will expect…
- You are given the Open English Wordnet (OEWN) database in SQLite format, located at /app/oewn.sqlite.
- You are given the output file of a Raman Setup. We used it to measure some graphene sample.
- Write a JSON file called /app/re.json that is a list of regex, replacement pairs. When executed in order with the Python code
- Write a regex expression that matches dates in the format YYYY-MM-DD appearing in lines that contain an IPv4 address in a log file.
- Help me create two scripts for managing the resharding of my dataset:
- You are given datasets /app/train X.csv, /app/train y.csv, /app/test X.csv, /app/meta public.json; and a R script /app/gp rstan.R.
- I have annotated histopathology slides with cell masks. The problem is that some of the masks
- Please help sanitize my github repository "dclm" of all API keys.
- Write a file eval.scm that is a metacircular evaluator capable of interpreting the scheme-like language implemented by interp.py.
- You are provided with a Turtle file at /app/university graph.ttl containing both the ontology and the instance data for a knowledge graph…
- I have a sqlite database in /app/trunc.db that was corrupted through binary truncation. Recover as many of the rows as possible, and…
- Compile SQLite in /app/sqlite with gcov instrumentation and make it available in the PATH.
- Implement pipeline parallel training for the LLaMA model using PyTorch. Create the file /app/pipeline parallel.py
- Implement tensor parallelism for linear layers using PyTorch.
- Please train a fasttext model on the yelp data in the data/ folder.
- Can you tune this MuJoCo model file (mjcf) such that it takes 60% of the original time or less to simulate the same scene for a total of…
- Write a script, named jump analyzer.py, and place it in /app/jump analyzer.py . The script analyzes MP4 videos of hurdle jumpers and…
- Analyze the following program that contains a secret key. The goal is to extract this secret key and save it to /app/results.txt .
- Write a CoreWars program that achieves a winning average against five classic opponent warriors.
- I have a decompressor in /app/decomp.c. It reads compressed data from stdin and writes the decompressed data to stdout. I also have a file…