Amazon Q is like pretty bad

Posted on Thursday, Dec 14, 2023 by Ned Bellavance

Featured in this episode of Tech News of the Week

I don’t want to belabor the point. If you want to read an absolutely vicious and accurate takedown of Amazon Q, check out Corey Quinn’s post. He helpfully notes that Amazon Q has been integrated into everything AWS, gives bad and often wrong answers, is outperformed by ChatGPT, and serves to illustrate just how much Amazon dropped the ball on this one.

But I didn’t just want to take his word for it, so I spun up Amazon Q in the AWS console and asked it a few questions. First up, how many NAT gateways should I have in a VPC with two public subnets? The correct answer, well it depends, but one or two would be acceptable. Q’s answer? It assumed I was having network connectivity problems and pointed me at the network connectivity analyzer. That’s, um, wrong.

Next, I asked how many root users I should have on an AWS account. It correctly answered that you can only have one and that you shouldn’t use it aside from initial setup tasks. Well done.

Lastly, I asked it, how can I connect my VPC to an Azure Vnet? To which it said, “Sorry, I can’t answer that question.” Once again reinforcing AWS’ grand tradition of pretending other cloud providers don’t exist. So, I guess that’s correct “from a certain point of view?”

I asked the same set of questions to ChatGPT, and… it fucking nailed all three of these questions. It even gave me step-by-step directions for creating VPN gateways on AWS and Azure to connect the two, and provided guidance on when to choose a NAT instance over a NAT gateway.

I get that Q is still in preview and all, but there is just no contest between the two. There’s more to the story involving possible data leakage, and maybe we’ll expand on that next week (wink, wink, nudge, nudge, know what I mean?)