Proof of Work

Proof of Work is a way one computer can verify that another computer has used some speficied amount of computational power. It was first proposed in 1993 as a way to deter internet attackers; to describe it as a non-technical metaphor, it’s very similar to if you came to me asking for something and I tasked you with bringing me back a specific amount of a thing.

How it proves you did the work yourself is by specifying something very specific, in my example, such as a box with specific and unusual dimensions and made out of a specific material. With computers, it’s a form of setting parameters and having the other computer calculate a result in such a way that the first computer can verify the result. The details aren’t importart; what’s important is that, short of quantum computing shaking things up, there are known mechanisms for doing this sort of “I declare, you perform, I verify” sequences. Hence the term “Proof of Work”. The work being done is arbitrary and usually useless – its entire purpose is for verification, and after being verified, is discarded. The only purpose it serves is to prove that some amount of calculation has been done.

Proof of work was initially most widely employed in Bitcoin. Bitcoin uses proof of work in mining, the process of discovering new blocks, which are the “gold” of Bitcoin. Bitcoin was designed so that blocks, just like IRL gold, are finite, making them a scarce resource.

Mining – proof-of-work – takes energy. It’s the whole point of proof-of-work – to cost CPU cycles, which has a real-world cost, in electricity, because computing uses energy. If POW was free, if it had no cost, it would not be useful. This very attribute of costing is exactly what makes it useful. It’s also one of the biggest criticisms of cryptocurrencies. It’s certainly not the only criticism, but it’s one of the first criticisms leveled at Bitcoin. The giant mining server farms, the massive energy use. It’s a very real, and valid criticism. It produces nothing but a block of data which has no intrinsic value or purpose.

LLMs are currently under criticism for burning huge amounts of energy, but at least they are doing work providing services. The actual value of which is questioned, but it’s unarguably more than producing results which are immediately discarded. Energy use by data centers is a significant percentage of global energy demand, and consequently, global warming.

Recently a project called Anubis has been gaining popularity. It’s a project that implements proof-of-work coming full circle back to the original purpose for which proof-of-work was originally proposed: to disincentivize undesireable use of resources, and while it’s not specifically only for LLM bots scraping the web for training data, LLM bots are certainly one of the prime motivators. Search engine bots are also “bad actors” Anubis is designed to hinder, but tools like Anubis really started getting popular in response to LLMs. Proof-of-work, whether for mining cryptocurrencies or for hindering web scraping bots, is fundamentally and by definition doing work for the sake of doing work. It’s performing calculations for the sole purpose of burning energy.

Anubis (and tools like it) don’t have a huge cost per client; the cost is in the scale. Anubis doesn’t prevent bots; it makes it more expensive to use bots. This can only be expensive if it’s at scale, and tools like Anubis do not discriminate between users or bots: perhaps Anubis’ POW consumes only 1 watt of electricity; the idea is that bot farms now have to spend more dollars scraping data because of Anubis than they did before – that’s the disincentivize part. However, this applies equally well to a million human visitors to a web site – each only burns one watt, but together – because of Anubis – they burn a million watts. The irony is that many of the same people who (justifiably) criticize LLM data centers and cryptocurrency mining farms are employing tools which do exactly the same thing they’re criticizing. It’s especially ironic because Anubis is at least partially targetted at LLM harvesters: it employs a mechanism that burns energy just to prove energy was burned, to stop bots harvesting data for use in something people use.

Anubis isn’t the only tool employing POW to deter bots, it’s just a popular one that’s becoming increasingly popular. It’s also trying to address an issue people have with an increasingly hostile web, one that’s infested with bots scraping data relentlessly, costing web hosters money and providing nothing to them in return. LLMs are also fraught with problems, from inefficiency to profiteering on other people’s work with not even the good neighbor effort of giving credit to creators. I do not suggest that Anubis is a bad thing, and I certainly won’t say that LLMs – or any other scraper bots – are good. There’s a danger there, though, that people are weilding a tool without understanding the nature of the tool, and the real cost; and if the motivation is to penalize high energy users, then Anubis (and tools like it) is the wrong tool.

There are alternatives; honeypots, or simply adding a per-request delay, can make harvesting more expensive for bot farms without adding processing cost. There’s an established mechanism, robots.txt, which – if present – tell bots that the site doesn’t want to be scraped, but it’s often ignored As is usually the case with “self-regulation” solutions by bot farms. Honeypots can be recognized and short-circuited by sufficiently smart bots, and per-request delays may not be enough disincentive to prevent scraping.

I don’t have a better solution, but I question whether the cure is worth the cost, and I know there’s an unconcious hypocracy Can hypocracy be unconcious, or does it imply concious decision? in using POW-based firewalls to punish environmentally harmful industries. Mostly, though, I think if people are going to choose tools, they should understand the impact, and I’d hope FOSS projects like Anubis would clearly state this impact.