Amazon plans ‘deep dive’ internal meeting to address outages

Dave Treadwell, a top executive overseeing the technical foundations of Amazon’s website, told employees that the company’s “This Week in Stores Tech,” or TWiST, meeting would be a “deep dive” on “some of the issues that got us here.” The meeting was scheduled to begin at 12:30 p.m. ET.

“Folks – as you likely know, the availability of the site and related infrastructure has not been good recently,” Treadwell, senior vice president of eCommerce Foundation, wrote in a note to employees viewed by CNBC. He added that he was shifting the focus of the meeting “given the incidence of Sev 1s,” referring to high-severity incidents that cause outages or degraded performance of critical systems.

Amazon experienced four such incidents in a week, Treadwell said and noted the deep dive is necessary to “regain our strong availability posture.”

The Financial Times was first to report on the memos. An Amazon spokesperson said TWiST is a regular weekly meeting where retail tech leaders review the performance of store operations.

“As part of normal business, the meeting will include a review of the availability of our website and app as we focus on continual improvement,” the spokesperson said in a statement.

Earlier on Tuesday, an internal document indicated that “GenAI-assisted changes” involving “GenAI tools” were a factor in a “trend of incidents” since the third quarter. However, the bullet point referencing GenAI was deleted before the meeting, according to an updated version of the document viewed by CNBC and a person familiar with the matter who asked not to be named because of confidentiality.

After initial publication of this story, an Amazon spokesperson said a single incident was related to AI and none of the incidents involved AI-written code.

The meeting comes after Amazon’s online store malfunctioned for some users last week. For roughly six hours on Thursday, website and app users were unable to check out, access account information or view product prices. Amazon said in a statement that the issues were related to a “software code deployment.”

Amazon and its hyperscaler rivals are ramping up spending on infrastructure to manage soaring demand for artificial intelligence services, which require increasing amounts of computing power. In its earnings report last month, Amazon said it expects $200 billion in capital expenditures this year, more than any of its tech peers.

As it boosts AI spending, Amazon is simultaneously continuing to slash jobs. The company in January laid off about 16,000 corporate workers, after a prior round of mass job cuts in October, when roughly 14,000 roles were eliminated. Amazon also laid off more than 27,000 employees between 2022 and 2023.

Treadwell acknowledged that “best practices and safeguards” around generative AI usage haven’t been fully established yet.

Amazon plans to “reinforce” various safeguards to prevent further issues, including requiring additional review of “GenAI-assisted” production changes, according to the memo.

“We are implementing temporary safety practices which will introduce controlled friction to changes in the most important parts of the Retail experience, in parallel we will invest in more durable solutions including both deterministic and agentic safeguards,” Treadwell wrote.

Amazon Web Services has also been hit with several outages in recent months, though the company said Tuesday that the cloud group is not involved in the incidents referenced by Treadwell.

AWS was hit in December in an incident that took down a cost management feature for an extended period of time, according to several reports. The FT reported the issue occurred after engineers allowed its Kiro AI coding tool to make changes.

Amazon said in a statement at the time that the outage was the result of “user error” and not AI.

Update: This story has been updated to reflect changes Amazon made in an internal document after initial news reports were published.

WATCH: Amazon’s a cheap stock and there’s opportunity