We're moving progressively in the direction of "pages can't be served for free anymore". Which, I don't think is a problem, and in fact I think it's something we should have addressed a long time ago.
Cloudflare only needs to exist because the server doesn't get paid when a user or bot requests resources. Advertising only needs to exist because the publisher doesn't get paid when a user or bot requests resources.
And the thing is... people already pay for internet. They pay their ISP. So people are perfectly happy to pay for resources that they consume on the Internet, and they already have an infrastructure for doing so.
I feel like the answer is that all web requests should come with a price tag, and the ISP that is delivering the data is responsible for paying that price tag and then charging the downstream user.
It's also easy to ratelimit. The ISP will just count the price tag as 'bytes'. So your price could be 100 MB or whatever (independent of how large the response is), and if your internet is 100 mbps, the ISP will stall out the request for 8 seconds, and then make it. If the user aborts the request before the page loads, the ISP won't send the request to the server and no resources are consumed.
> We're moving progressively in the direction of "pages can't be served for free anymore". Which, I don't think is a problem, and in fact I think it's something we should have addressed a long time ago.
I agree, but your idea below that is overly complicated. You can't micro-transact the whole internet.
That idea feels like those episodes of Star Trek DS9 that take place on Feregenar - where you have to pay admission and sign liability wavers to even walk on the sidewalk outside. It's not a true solution.
I agree that end-users cannot handle micro transactions across the whole internet. That said, I would like to point out that most of the internet is blanketed in ads and ads involve tons of tiny quick auctions and micro transactions that occur on each page load.
It is totally possible for a system to evolve involving tons of tiny transactions across page loads.
The presented solution has invisible UX via layering it into existing metered billing.
And, the whole internet is already micro-transactioned! Every page with ads is doing a bidding war and spending money on your attention. The only person not allowed to bid is yourself!
> We're moving progressively in the direction of "pages can't be served for free anymore". Which, I don't think is a problem, and in fact I think it's something we should have addressed a long time ago.
But it's done through a bait and switch. They serve the full article to Google, which allows Google to show you excerpts that you have to pay for.
It would be better if Google shows something like PAYMENT REQUIRED on top, at least that way I know what I'm getting at.
> They serve the full article to Google, which allows Google to show you excerpts that you have to pay for.
I'm old enough to remember when that was grounds for getting your site removed from Google results - "cloaking" was against the rules. You couldn't return one result for Googlebot, and another for humans.
No idea when they stopped doing that, but they obviously have let go of that principle.
I remember that too, along with high-profile punishments for sites that were keyword stuffing (IIRC a couple of decades ago BMW were completely unlisted for a time for this reason).
I think it died largely because it became impossible top police with any reliability, and being strict about it would remove too much from Google's index because many sites are not easily indexable without them providing a “this is the version without all the extra round-trips for ad impressions and maybe a login needed” variant to common search engines.
Applying the rule strictly would mean that sites implementing PoW tricks like Anubis to reduce unwanted bot traffic would not be included in the index if they serve to Google without the PoW step.
I can't say I like that this has been legitimised even for the (arguably more common) deliberate bait & switch tricks is something I don't like, but (I think) I understand why the rule was allowed to slide.
“Free” could have a number of meanings here. Free to the viewer, free to the hoster, free to the creator, etc…
That content can't be served entirely for free doesn't mean that all content will require payment, and so is subject to issues with payment processors, just that some things may gravitate back to a model where it costs a small amount to host something (i.e. pay for home internet and host bits off that, or you might have VPS out there that runs tools and costs a few $ /yr or /month). I pay for resources to host my bits & bobs instead of relying on services provided in exchange for stalking the people looking at them, this is free for the viewer as they aren't even paying indirectly.
Most things are paid for anyway, even if the person hosting it nor my looking at it are paying directly: adtech arseholes give services to people hosting content in exchange for the ability to stalk us and attempt to divert our attention. Very few sites/apps, other than play/hobby ones like mine or those from more actively privacy focused types, are free of that.
That's already a deep problem for all of society. If we don't want that to be an ongoing issue, we need to make sure money is a neutral infrastructure.
It doesn't just apply to the web, it applies to literally everything that we spend money on via a third party service. Which is... most everything these days.
My first reaction: This solution would basically kill what little remaining fun there is to be had browsing the Internet and all but assure no new sites/smaller players will ever see traffic.
Curious to hear other perspectives here. Maybe I’m over reacting/misunderstanding.
Depending on the implementation (a big if) it would help smaller websites, because it would make hosting much cheaper. ISPs don’t choose what sites users visit, only what they pay. As long as the ISP isn’t giving significant discounts to visiting big sites (just charging a fixed rate per bytes downloads and uploaded) and charging something reasonable, visiting a small site would be so cheap (a few cents at most, but more likely <1 cent) users won’t weigh cost at all.
But users depend on major sites like google [insert service] still and will prioritize their usage accordingly like limited minutes and texts back in the day, right?
Networking is so cheap, unless ISPs drastically inflate their price, users won’t care.
The average American allegedly* downloads 650-700GB/month, or >20GB/day. 10MB is more than enough for a webpage (honestly, 1MB is usually enough), so that means on average, ISPs serve over 2000 webpages worth of data per day. And the average internet plan is allegedly** $73/month, or <$2.50/day. So $2.50 gets you over 2000 indie sites.
That’s cheap enough, wrapped in a monthly bill, users won’t even pay attention to what sites they visit. The only people hurt by an ideal (granted, ideal) implementation are those who abuse fixed rates and download unreasonable amounts of data, like web crawlers who visit the same page seconds apart for many pages in parallel.
Yeah same reaction here - there's no world in which ISP's would agree to this and even if they did I don't want to add them to my list of utilities I have to regularly fight with over claimed vs. actual usage like I do with my power/water/gas companies.
Why would I pay for a page if I don't know if the content is what I asked for? How much are you going to pay? How much are you going to charge? This will end up in SEO hell, especially with AI-generated pages farming paid clicks.
I do believe we will end there eventually, with the emerging tech like Brazil’s and India’s payment architectures it should be a possibility in the coming decades
I think value is not proportional to bytes - an AI only needs to read a page once to add it to its model, and then served the effectively cached data many times.
Sadly development along these lines has not progressed. Yes, Google Cloud and other services may return it and require some manual human intervention, but I'd love to see _automatic payment negotiation_.
I'm hopeful that instant-settlement options like Bitcoin Lightning payments could progress us past this.
Your theory does not match the practice of Cloudflare.
Whatever method is used by Cloudflare for detecting "threats" has nothing to do with consuming resources on the "protected" servers.
The so-called "threats" are identified in users that may make a few accesses per day to a site, transferring perhaps a few kilobytes of useful data on the viewed pages (besides whatever amount of stupid scripts the site designer has implemented).
So certainly Cloudflare does not meter the consumed resources.
Moreover, Cloudflare preemptively annoys any user who accesses for the first time a site, having never consumed any resources, perhaps based on irrational profiling based on the used browser and operating system, and geographical location.
As time passes I’m more certain in the belief that the internet will end up being a licensed system with insanely high barriers to entry which will stop your average dev from even being able to afford deploying a hobby project on it.
Your idea of micro transacting web requests would play into it and probably end up with a system like Netflix where your ISP has access to a set of content creators to whom they grant ‘unlimited’ access as part of the service fee.
I’d imagine that accessing any content creators which are not part of their package will either be blocked via a paywall (buy an addon to access X creators outside our network each month) or charged at an insane price per MB as is the case with mobile data.
Obvious this is all super hypothetical but weirder stuff has happened in my lifetime
That’s fine. The point for website owners isn’t to make money, it’s to not spend money hosting (or more specifically, to pay a small fixed rate hosting). They want people to see the content; if someone makes the content more accessible, that’s a good thing.
You ignore the issue of motivation. Most web content exists because someone wants to make money on it. If the content creator can't do that, they will stop producing content.
These AI web crawlers (Google, Perplexity, etc) are self-cannibalizing robots. They eat the goose that laid the golden egg for breakfast, and lose money doing it most of the time.
If something isn't done to incentivize content creators again eventually there will be only walled-gardens and obsolete content left for the cannibals.
AFAIK, currently creators get money while not charging for users because of ads.
While I don’t blame creators for using ads now, I don’t think they’re a long-term solution. Ads are already blocked when people visit the site with ad blockers, which are becoming more popular. Obvious sponsored content may be blocked with the ads, and non-obvious sponsored content turns these “creators” into “shills” who are inauthentic and untrustworthy. Even without Google summaries, ad revenue may decrease over time as advertisers realize they aren’t effective or want more profit; even if it doesn’t, it’s my personal opinion that society should decrease the overall amount of ads.
Not everyone creates only for money, the best only create for enough money to sustain themselves. A long-term solution is to expand art funding (e.g. creators apply for grants with their ideas and, if accepted, get paid a fixed rate to execute them) or UBI. Then media can be redistributed, remixed, etc. without impacting creators’ finances.
The reason why that didn’t work was because regulations made micropayments too expensive, and the government wants it that way to keep control over the financial system.
Cloudflare only needs to exist because the server doesn't get paid when a user or bot requests resources. Advertising only needs to exist because the publisher doesn't get paid when a user or bot requests resources.
And the thing is... people already pay for internet. They pay their ISP. So people are perfectly happy to pay for resources that they consume on the Internet, and they already have an infrastructure for doing so.
I feel like the answer is that all web requests should come with a price tag, and the ISP that is delivering the data is responsible for paying that price tag and then charging the downstream user.
It's also easy to ratelimit. The ISP will just count the price tag as 'bytes'. So your price could be 100 MB or whatever (independent of how large the response is), and if your internet is 100 mbps, the ISP will stall out the request for 8 seconds, and then make it. If the user aborts the request before the page loads, the ISP won't send the request to the server and no resources are consumed.