Home Technology Apple Inc. Said To Be Running Suspicious Web Crawler [REPORT]

Apple Inc. Said To Be Running Suspicious Web Crawler [REPORT]

When you purchase through our sponsored links, we may earn a commission. By using this website you agree to our T&Cs.

Apple Inc. (NASDAQ:AAPL) seems to be running an automated web crawler that’s crawling HTML websites for an unknown purpose. The crawler is apparently written in the Go programming language and turns off sometimes.

Developer discovers Apple crawler

Apple Insider points to a tweet from developer Jan Moesen, who seems to be the first to have run across the mysterious crawler and asks whether anyone knows why Apple is running it.

Apparently it comes from an IP address that starts with the number 17 and that all IP addresses come from Apple’s servers. The developer found evidence of the web crawler dating back to Oct. 15, and he’s not the only one to notice it. Others have said that there seems to be quite a bit of traffic from Apple’s mysterious web crawler.

What is the crawler for?

The automated bot only requests information from HTML pages and doesn’t actually seem to get into the CSS part of the website or the JavaScript or images that are on the pages. According to Moesen, the bot gives an identification of Mozilla/ 5.0 (compatible; Fetcher/0.1).

According to Apple Insider, it’s unclear whether the web crawler is officially from Apple or whether it’s some sort of unofficial side project being run by an employee at the company. The website suggests that the bot could have something to do with Apple’s improvements in its Spotlight feature, which is in both iOS 8 and OS X Yosemite.

Both of them received updates recently that enable users to directly access online content without using third party search tools like Google Inc (NASDAQ:GOOGL) (NASDAQ:GOOG)’s Search or Microsoft Corporation (NASDAQ:MSFT)’s Bing search engine. Apple has been trying to get away from using competitors to bring up information for searches or queries done through its Siri digital assistant. Currently Apple uses Bing as its official search partner for Siri.

Our Editorial Standards

At ValueWalk, we’re committed to providing accurate, research-backed information. Our editors go above and beyond to ensure our content is trustworthy and transparent.

Michelle Jones

Want Financial Guidance Sent Straight to You?

  • Pop your email in the box, and you'll receive bi-weekly emails from ValueWalk.
  • We never send spam — only the latest financial news and guides to help you take charge of your financial future.