osolmaz/aacr-bench-harbor
aacr-bench-harbor: Harbor dataset on Hugging Face with 190 tasks. See what each task asks, how it is graded and what it runs in, then run an agent on it.
Tasks
- Review the pull-request change from merge-base commit 67d55b967da90edfc8ccfdab7f914454bb908e57 to head commit…
- Review the pull-request change from merge-base commit d81ba177cc8b3da5f18190c18f2de55b80bb5895 to head commit…
- Review the pull-request change from merge-base commit e267283fa9981f6f1e7af73d77437ad0bec5a255 to head commit…
- Review the pull-request change from merge-base commit 041c7a5e63d5559ce5571a30e177c072b1b357b2 to head commit…
- Review the pull-request change from merge-base commit b432e367427f1f9fe0f0a5800e31e496f00cd38d to head commit…
- Review the pull-request change from merge-base commit db36a92c02b83f2e6477a5a55fc061319f7cc6a3 to head commit…
- Review the pull-request change from merge-base commit baa848b8d38187ce6b24a57cfadf1ea180209711 to head commit…
- Review the pull-request change from merge-base commit 7c369e84052aaee15a6dedf8928b5af4c869037b to head commit…
- Review the pull-request change from merge-base commit 841251d09eb6d512cf27806b5ed089a9ce873b68 to head commit…
- Review the pull-request change from merge-base commit b50bb2216721b82d81e0fda8fb50a31266e09ce4 to head commit…
- Review the pull-request change from merge-base commit e85a547378b236c9788802d28e816f10940a7e53 to head commit…
- Review the pull-request change from merge-base commit 01abdc414b4bea9da34d10c2fb5383c870396f50 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 227ea3e39be11684d1720e9bf604539541f21a36 to head commit…
- Review the pull-request change from merge-base commit 66394a7495d9ce9bb9defe27185358d402cef8d2 to head commit…
- Review the pull-request change from merge-base commit d144f444affac79d6b43b45fc1d68c46a7766750 to head commit…
- Review the pull-request change from merge-base commit dacc8ebb0dac5bbfce5a7541e7fc70f26f7d5065 to head commit…
- Review the pull-request change from merge-base commit 6a1676ef28d3522eb4f74ad3423ad09752c232dc to head commit…
- Review the pull-request change from merge-base commit 4ff4869e3bdd97f7de69f51f994fc7089c6fd412 to head commit…
- Review the pull-request change from merge-base commit 280ecea7c6e906888b0cb3bd3a84fa5c23a43cb9 to head commit…
- Review the pull-request change from merge-base commit fe23559f719b77a1f8395ebcaafd778ae7183b6e to head commit…
- Review the pull-request change from merge-base commit 76957a2de2b46b54f45f87d3f2e814e774d4f85a to head commit…
- Review the pull-request change from merge-base commit 1b7dc9c25090cf8cd75eb92b951cc74033eccab0 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 9e13a594de549fd477e1b90be9fbd57068889548 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 23a0d4164651f039da6c730dac07a4a47689513e to head commit…
- Review the pull-request change from merge-base commit e84f2ff962f708a68390bc855cf9cd07228a3637 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit a1bf1f95a30ca95581d9811488636aec42dd2f80 to head commit…
- Review the pull-request change from merge-base commit 9dc021a881852866d3ef5c9636b929a9e1b68f22 to head commit…
- Review the pull-request change from merge-base commit 8cdc4a936d64483f3123509ab3688f452fcd6d91 to head commit…
- Review the pull-request change from merge-base commit 2cddbf0821c0e9b74993b2e2b546af2b0d57aa2c to head commit…
- Review the pull-request change from merge-base commit fca304debf5a4e6c71f8def10ac41cf69519fde3 to head commit…
- Review the pull-request change from merge-base commit c7b25784b141a298d8ee5f0d830a29da6e8a3d54 to head commit…
- Review the pull-request change from merge-base commit 8627b84646d4ffa6a36c54e9cd5edec11035d945 to head commit…
- Review the pull-request change from merge-base commit 2125ab9048023375ae49aa66f65be384f128d4a3 to head commit…
- Review the pull-request change from merge-base commit 0f78c420bd314d078cb8d58e4fb55572d2b090aa to head commit…
- Review the pull-request change from merge-base commit 1faab74be397e7577708fa64f0ec6bc5c6057b78 to head commit…
- Review the pull-request change from merge-base commit 3fe06183a8d3ef32de0d16863d232d5652b1fe9b to head commit…
- Review the pull-request change from merge-base commit f83c97fa0b91f83bd20511d1abf235dc48b596e2 to head commit…
- Review the pull-request change from merge-base commit 1b5d54eb39283b0d028841bd4a73fe8ebaa4abd7 to head commit…
- Review the pull-request change from merge-base commit e7d49f52363753a4faa2a22538eace7a923382ce to head commit…
- Review the pull-request change from merge-base commit 4f030efcd50781fe6e624dc08046b834786edb4a to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit d3f2956116eea89543dc4979bd3cdfc5c54d8bd7 to head commit…
- Review the pull-request change from merge-base commit dbbcb09244cd65b52c13f78e500deb5fdcd3edd9 to head commit…
- Review the pull-request change from merge-base commit 4ac8d55ff7c5612413e4a45e14a2b39f3e85f3a9 to head commit…
- Review the pull-request change from merge-base commit 1d9acc1a421e83776512b0606c6bd2bb95df750c to head commit…
- Review the pull-request change from merge-base commit 6b27e420fed187de0add194eb32bb940688af26f to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 930b4ab9957ed4a6376bcac7ba17f7bb2e82d125 to head commit…
- Review the pull-request change from merge-base commit 79297438ed2b6f2fb5315508157d6584c84f1760 to head commit…
- Review the pull-request change from merge-base commit b8d741765f1a38d07bb503a6129899ac13242222 to head commit…
- Review the pull-request change from merge-base commit 63258f49bfc4a9895f40fac8da7ae676a1840aeb to head commit…
- Review the pull-request change from merge-base commit 971c87dc70b66d81faba990854d0c084fbe34823 to head commit…
- Review the pull-request change from merge-base commit bede881840643d36010595d19db22803824723d0 to head commit…
- Review the pull-request change from merge-base commit 22b7403cd1beb2aa95bec223ddd6f328c18b7f8c to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 924a8da940ec0acefeb6d43e74f5669a8704d159 to head commit…
- Review the pull-request change from merge-base commit 2a51a997ebd60d9e093ecb58dfb791361e87c161 to head commit…
- Review the pull-request change from merge-base commit 2980f27779cf37a9656b25418a3c5cfca989e244 to head commit…
- Review the pull-request change from merge-base commit 8e9a5fc6c1a6252ca1727ab8fe0d4ee13f2568ec to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit e4a7ecf0e65574522eb9fbe70b7339f41de0bbe2 to head commit…
- Review the pull-request change from merge-base commit ce4d9d08df41142ff219bb3dc44968084eaae0dc to head commit…
- Review the pull-request change from merge-base commit 8b44845f2bfab244f650d38e1b0ac00e733612e7 to head commit…
- Review the pull-request change from merge-base commit 3b502359353e2a74dee8a8bcfed5750b69f32cdc to head commit…
- Review the pull-request change from merge-base commit fbb1225f909bd7de602d9cd1b540d8099ca3429a to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 855ad6c70904f911b2ce7114dd1a5bc24da98bd1 to head commit…
- Review the pull-request change from merge-base commit 53c4c2ef50870fd7a540b9511548b97a73f399b2 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 9a6a98f824436656cae242ae26b043f78411ebba to head commit…
- Review the pull-request change from merge-base commit d84d9c9bd19eff174a6a5de29282be5808ff1be4 to head commit…
- Review the pull-request change from merge-base commit ce368cb7a0dd6a3f57e5cabc3e7745c5669683a9 to head commit…
- Review the pull-request change from merge-base commit 730e07db5d2acafe7723eb89c83641375c375bc3 to head commit…
- Review the pull-request change from merge-base commit f3e1e6cec065f0295e38d36a68e0390f1b977a32 to head commit…
- Review the pull-request change from merge-base commit 567a9ca6e318ed76523533cdfc47c6199f1e5ac6 to head commit…
- Review the pull-request change from merge-base commit 30116f05e9b88763fabc862af0a0f3a8a83a0c5b to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 4d679f47e1525868cfeee249f981b18f054a1824 to head commit…
- Review the pull-request change from merge-base commit 717443dc91dbf04169789d41bd4998a6ac858a14 to head commit…
- Review the pull-request change from merge-base commit 92564a7accc4dc1e7570c3dc044b6250510e1177 to head commit…
- Review the pull-request change from merge-base commit 7ffe8038efaa5bf263a2a933819bcd4badd37dc2 to head commit…
- Review the pull-request change from merge-base commit 273e74c09da89492d16fc076cfbff4d043eafa4c to head commit…
- Review the pull-request change from merge-base commit 0d65baf9283138da56cdf08b00058ab3cf8cbaf9 to head commit…
- Review the pull-request change from merge-base commit 3af4913ef3f00de71744de551a568aa713a3beec to head commit…
- Review the pull-request change from merge-base commit bd0f085ae896e97373b451180f70ff551142fd7e to head commit…
- Review the pull-request change from merge-base commit 0534ca74d4e67b00ed51e9f8bf568efc3ecaf834 to head commit…
- Review the pull-request change from merge-base commit 570b0086b606d2a6d55a2bdcd08b8a91c8092c8d to head commit…
- Review the pull-request change from merge-base commit e0fc8734bfff7a41baeeacda3790a91c3bfd93bc to head commit…
- Review the pull-request change from merge-base commit c3be74c450ba7d57ccfc9f6c0463478e579682a0 to head commit…
- Review the pull-request change from merge-base commit f659ef4b7aeb20f2fe433abb7fc6f7b4d95bb0ea to head commit…
- Review the pull-request change from merge-base commit 6ba94ac2f2f78c871f56874a7c2ff61b2ff4a06b to head commit…
- Review the pull-request change from merge-base commit 92f0973a46acd813fd595fb6fdbf8cf143992b51 to head commit…
- Review the pull-request change from merge-base commit d0014bdf94feecf84b375499a8e48bb9fa47a8b3 to head commit…
- Review the pull-request change from merge-base commit 15d431ba6a9caf6ba8f4a97ea357bfcd00c7245b to head commit…
- Review the pull-request change from merge-base commit 409310aae90c12ad8a85320c4a71b2108485edc9 to head commit…
- Review the pull-request change from merge-base commit b271cc34b3e5fa24ca067d662b2b0ac4fb561c51 to head commit…
- Review the pull-request change from merge-base commit e2b66628f4728aeda0c1db5aa8e75d5d6729f3db to head commit…
- Review the pull-request change from merge-base commit 487aed419e393122558770b9df0c65e18412cc88 to head commit…
- Review the pull-request change from merge-base commit 9f334c1ebcf0e8ee15ae2af608fb8f27fab2a6b2 to head commit…
- Review the pull-request change from merge-base commit 34dd99d8050a84f0478cdb4fa0f0cc83608e52d9 to head commit…
- Review the pull-request change from merge-base commit 24beaa224a08301b40ee18c617cbf116049eb3d5 to head commit…
- Review the pull-request change from merge-base commit e36ee1d6ba2f99402ebd4a71e356e3e071f0557c to head commit…
- Review the pull-request change from merge-base commit 2e09777790d4a64d746175f94f8e2861844aaede to head commit…
- Review the pull-request change from merge-base commit dc64d81dd0f1d9832df6e69883a33c95dff5398d to head commit…
- Review the pull-request change from merge-base commit 2c3b4ff92940db9b9349b6edd6c6b52bbc8ec53b to head commit…
- Review the pull-request change from merge-base commit 53795fa758baeb9ad1a398ed3d3027bbec339a39 to head commit…
- Review the pull-request change from merge-base commit 92c96033f2c32012b9303c4fa4b3501d3597e5fa to head commit…
- Review the pull-request change from merge-base commit b3b30a8620dcce0b0d5caee429d07789afbad147 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 725f1e40f0f8195833a4c1ecade693aa3a0d6ed0 to head commit…
- Review the pull-request change from merge-base commit 4b0c42e52076fb7cdca084a1ab06d46f87fecff1 to head commit…
- Review the pull-request change from merge-base commit 22e24fa718a511e657c7c120721fa129f3c3b677 to head commit…
- Review the pull-request change from merge-base commit e15fddd3aaac4b1773ea09f60d63b81591894f7a to head commit…
- Review the pull-request change from merge-base commit 3d5c00c613cbeb162044778ec2ad15b3032e4d02 to head commit…
- Review the pull-request change from merge-base commit 1472088e7026f8d87d3c38dfc1bc7e023329a039 to head commit…
- Review the pull-request change from merge-base commit 6d94b16fd5f4f72bb741f9b900b83179c8ab46c9 to head commit…
- Review the pull-request change from merge-base commit ab34ea5a2684766e510de1daf8ae44c41b6af704 to head commit…
- Review the pull-request change from merge-base commit 71303b41bf89c4bb68dbe1bed12fd2ee1dc0dd41 to head commit…
- Review the pull-request change from merge-base commit 73816f6f8c07708c474394c610f4f32f88be0e7b to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit c31634e52d719fdf60694f47812e2f007022ae26 to head commit…
- Review the pull-request change from merge-base commit 3f13398673c810ce5b3f7b21a7b89753f5e9ba97 to head commit…
- Review the pull-request change from merge-base commit e150297232acb283717f5d22e49e39c80ebb5667 to head commit…
- Review the pull-request change from merge-base commit 344ba9fe99b487f7f5164900679d56acf2d8bff2 to head commit…
- Review the pull-request change from merge-base commit 2e3e1ea29406128732c97ece9a92814843a051c1 to head commit…
- Review the pull-request change from merge-base commit e1474c1f3020e2615c438d82b29f48626f456a9f to head commit…
- Review the pull-request change from merge-base commit 78a67933e3acb418a21f6794a0d25aca519eaff9 to head commit…
- Review the pull-request change from merge-base commit a06d95b0fe0346851ada34b05d1af901b2e28ab3 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit 5e6bd1efcfb0020fc221d06492248db780d68eb1 to head commit…
- Review the pull-request change from merge-base commit da4e121142cd3f891785b573a194b9caaaa01d55 to head commit…
- Review the pull-request change from merge-base commit 8a218b02a52b13d0ca8264a4c897c6dbd0b2b4d5 to head commit…
- Review the pull-request change from merge-base commit d851a61fc472fd4e50f0d23b84bd6f102bec3a6e to head commit…
- Review the pull-request change from merge-base commit e4b64fba9c5c6ca97189f32c056d05c2b47cebd6 to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit a55ae5d6cdba0f96968b56e91a3999db40e31e61 to head commit…
- Review the pull-request change from merge-base commit 5ba7ee513751dd005379a06fb627303e6fe1b3dd to head commit…
- Review the pull-request change from merge-base commit 1fc010cb7bd1e17d94c1a33ba8dacdf9e34193cb to head commit…
- Review the pull-request change from merge-base commit 17d0b43abd9b20c7038d161bcaae81479a58c7ce to head commit…
- Review the pull-request change from merge-base commit de413e2112ff1ae80599f145b101354a1476976c to head commit…
- Review the pull-request change from merge-base commit 20edd44463c15baa305d8cb988f05a5600baa13b to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit cfea538f1cdb1a3db76915b485fbd315e092217c to head commit…
- Review the pull-request change from merge-base commit 131baabb7fc660b00470ff1c5981ca65ee58753d to head commit…
- The repository is in repo/ and checked out at the head commit. The merge base is available as Git ref aacr-base . Inspect the diff and any…
- Review the pull-request change from merge-base commit c3fff03945869778635d270085122fb01953d3b2 to head commit…
40 more tasks.