HF RL Explorer

Extend linkchecker GET fallback logic to handle Too Many Redirects Subject: linkcheck - fallback to GET…

Extend linkchecker GET fallback logic to handle Too Many Redirects Subject: linkcheck - fallback to GET…: a task in Lego-RL-SWE-Bench-Verified (Harbor dataset). Some websites will enter infinite redirect loops with HEAD requests. In this case, the GET fallback is ignored as the exception is of…

The task

Some websites will enter infinite redirect loops with HEAD requests. In this case, the GET fallback is ignored as the exception is of type `TooManyRedirects` and the link is reported as broken. This extends the except clause to retry with a GET request for such scenarios.

Part of Lego-X/Lego-RL-SWE-Bench-Verified.