I’ve been in the publishing industry for over 20 years. During that time, I’ve seen search engines change how content is discovered and ranked. But recently, something happened that I’ve never experienced before – one of our most established, content-rich websites was suddenly removed from Bing’s search index. And to be clear: Bingbot is still crawling the site. It just isn’t indexing it.
A 10-Year-Old Website, Still Crawled but No Longer Indexed
The site in question has been live for over a decade. It was built on original content, regularly updated, and technically optimized over the years. It’s not powered by AI or filled with fluff – it’s a real, human-created resource that served its niche very well. For years, it performed strongly in search, including on Bing.
Then, with no changes made to the site, we noticed something strange: Bing was no longer showing any of the site’s pages in its search results. We checked Bing Webmaster Tools – crawl stats showed that Bingbot was still visiting the site regularly. But indexed pages? Zero.
So let me repeat: Bingbot is actively crawling our website, yet none of the pages are being indexed.
No Clues from Bing Webmaster Tools
Naturally, I turned to Bing Webmaster Tools looking for answers. There were no errors. No spam warnings. No indexing restrictions in robots.txt or meta tags. The sitemap was accepted. Bingbot activity was clearly visible. Everything looked normal – except the complete absence of indexed pages.
I submitted a support ticket, hoping to get clarity. What I received was this vague, unhelpful message:
Thank you for your patience!
After further review, it appears that your site https://*****.com/ did not meet the standards set by Bing the last time it was crawled.
Bing constantly prioritizes the content to be indexed that will drive highest users satisfaction. Please review our Bing Webmaster Guidelines, to better understand criteria for most valuable content.
We hope the resolution provided has been able to fully address your issue. We will be closing this ticket. However, if you do have any follow up questions or concerns, please submit a feedback here.
Sincerely,
Bing Webmaster Support Team
This isn’t just our experience – countless other publishers report receiving this exact same response when contacting Bing Support with indexing concerns.
We’re Not Alone in This
After sharing the situation with a few industry peers, I quickly realized this isn’t an isolated case. Other long-standing, original-content websites – clean, white-hat, and compliant – are facing the same issue. Bingbot crawls them, but they’re not indexed. No errors. No warnings. Just silence.
This isn’t a glitch. It’s part of a bigger, more systemic shift.
The Real Problem: Crawling is Not the Same as Indexing Anymore
Historically, if a search bot crawled your site, you had a pretty good chance of being indexed – assuming the site was healthy. But that’s no longer the case.
In my view, the issue comes down to two major trends:
- Skyrocketing resource costs for indexing, especially with the rise of AI-driven content analysis.
- An overwhelming flood of new web pages, many of them generated by AI tools.
Today, indexing isn’t just about fetching and storing a page. It involves deeper processing – detecting originality, assessing trust, analyzing user value. That AI-driven processing requires 8 to 10 times more resources per page than traditional methods. Some insiders estimate that it could be as much as 15 to 18 times more, depending on the complexity of the analysis.
At the same time, the amount of content being published has exploded – mostly thanks to AI. It’s not uncommon to see thousands of low-effort articles created in minutes. This sheer volume has pushed indexing systems to their limits.
Search Engines Are Struggling to Keep Up
Both Bing and Google are now facing serious pressure to manage the explosion of online content. Scaling infrastructure to meet today’s indexing demands would require massive investment – more servers, more bandwidth, more energy. And that’s hard to justify in an environment where ad revenues are unpredictable and investor expectations are high.
While Google may have more resources overall, even it has started to slow down or de-prioritize indexing in certain areas. Bing, operating on a smaller scale, appears to be even more aggressive in filtering what makes it into its index.
So, in the face of limited resources and endless content, search engines are making choices. And one of those choices seems to be quietly excluding some websites from indexing, even if they’re well-built and still being crawled.
Who Pays the Price? Publishers Like Us
For independent publishers, being excluded from a search engine like Bing – even while being crawled – is deeply frustrating. We’ve put in the time, the resources, and the expertise to create quality content. We’re not mass-producing junk or gaming the system. We’re simply doing honest, meaningful publishing – and somehow, that’s no longer enough to earn a spot in Bing’s index.
The result? Publishers lose traffic. Some lose revenue. And a lot of valuable content becomes invisible.
A Dangerous Trend That Could Get Worse
Unless search engines like Bing drastically increase their indexing capabilities or change how they prioritize sites, this problem will continue – and likely grow. More legitimate websites will vanish from the index without explanation. And with them, huge amounts of useful, human-created information will quietly disappear from the web.
Even if the trend reverses someday, it may be too late for many sites. Once a domain is devalued and forgotten by search engines, climbing back is extremely difficult – sometimes impossible.
Conclusion: Crawling is Not the Same as Indexing
Our site is still being crawled by Bingbot. The technical setup is fine. The content is original. The site ranks well in Google. And yet, Bing refuses to index it – offering only a vague “changing priorities” excuse.
To me, the real issue is simple: Bing doesn’t have the indexing capacity to handle the modern web, especially not with the explosion of AI content and the demands of AI-based evaluation. So it’s silently filtering out sites – sometimes at random, sometimes by flawed criteria – just to keep up.
That means publishers like us suffer. It means users miss out on valuable, time-tested content. And it means a part of the open web is fading away, one deindexed site at a time.
We can only hope this changes before too much is lost. But it’s worth noting: just a few years ago, about 80% of web pages were invisible and unreachable through search engines. Today, that number is estimated to be around 96% – and it’s still rising. If this trend continues, most of the web will soon become completely disconnected from search, and the open internet as we know it may never be the same.