Generative AI Offerings from AWS

Last updated at 2023-12-07Posted at 2023-12-07

With generative AI being one of the hottest topics today, I was interested to see what kind of related services are offered. So, I decided to do some research, and check websites and some news articles, to write this very simple compilation of what I've found in the process.

This article is focused on AWS offerings, but I plan to do the same for Azure and GCP to check their differences.

Disclaimer: I currently don't work in the field of AI and I'm just a user of services such as ChatGPT and GitHub Copilot.

AWS Strategy

You can find a compreehensive article on how to choose a machine learning (ML) solution on AWS, what includes the generative AI subset. Also, this very recent blog post from AWS details the company vision for its generative AI endeavours.

Image Source

Like most providers, AWS strategy for generative AI and related technologies are divided into three tiers:

infrastructure to train and develop large language models (LLMs)
foundation LLMs and other tools to build applications
applications built on top of the other two layers

In the infrastructure layer, these companies have been making their own chips with the obvious goals of improving performance and energy efficiency.

Some of the initiatives in this field include the AWS Inferentia line of inference chips that powers the Amazon EC2 Inf2 instances, as well as the the AWS Trainium line of chips, optimized for ML training, that powers the Trn1 instances. AWS also introduced the Neuron SDK in 2019 to make the most of those chips.

Then, there's also Amazon SageMaker, a fully managed set of tools for building, training and deploying ML models. In some places, it's considered to be part of the middle layer of the stack, probably because the abstraction it provides that allows not only data scientists and developers, but also business and data analysts to work with machine learning. But, it's probably more adequate to position it in the bottom layer because of the need to rely on that layer for the ML related tasks.

Tools to Build Applications

Although it only became generally available in September of 2023, Amazon Bedrock is "the AWS tool" to build with LLMs and other foundation models (FMs), allowing companies to make use of the existing models instead of creating their own.

This is a fully managed service that provides a single API to use the different FMs provided by AWS. Options include very well known models such as Claude from Anthropic and Llama 2 by Meta. Since it's serverless, there's no need to manage any infrastructure and the single API implementation allows the underlying foundation model to be swapped with little effort. Also, those FMs can be customized with company data to optimize them for the required task.

Thanks to its flexible nature, Bedrock allows easy experimentation with different models in a standardized way and the use the most appropriate model for the task, as each model works best in a specific domain. Such domains include dialog, content generation, reasoning, image generation, summarization, etc.

Bedrock has a feature called "Agents" that allows generative AI applications to execute multistep tasks to solve business problems. Those agents can, for example, retrieve information from company data sources or through API calls to fulfill a request.

Also, Bedrock provides a preview feature called "Guardrails" that allows companies to customize safeguards to be applied to the responses from the underlying FM. With this, we can list a set of topics to be avoided within the context of our applications. For example, we can filter harmful contents such as hate and insults, as well as remove Personally Identifiable Information (PII) from the records of the interactions with customers.

Applications Built on Top of Generative AI

In the top of the stack offered by Amazon are the applications that leverage LLMs and other FMs. One of those applications is "CloudWhisperer", an AI tool to support software development and an alternative to GitHub Copilot.

One highly marketed feature is its customization capability that allows CodeWhisperer to be aware of a company's internal code base to improve the relevancy and usefulness of the code recommendations. Such feature is also expected to be offered by GitHub Copilot soon.

And we also have "Amazon Q", AWS's generative AI-powered assistant. It works similarly to the mainstream customer oriented AI assistants, but its scope is much narrower. AWS's focus is clearly on its use for work scenarios.

The assistant has different areas of expertise:

AWS: get answers and guidance regarding AWS services, suggestions for the best solution for a use case and help on optimizing resources and solving problems. We can interact with the assistant through the AWS Console, our IDE or chat apps like Slack or Microsoft Teams
Business: get answers based on company's data that may come from Microsoft 365, Salesforce, Slack, Gmail, etc
Amazon QuickSight: Q can be plugged into AWS's business intelligence service and it allows users to get insights from their data. For example, it can help explain the reasons for a recent change in a graph of number of orders
Amazon Connect: can help customer service agents to get information from the company's knowledge base to help solving customer issues faster
There's also an area of expertise to help solve problems related to supply chain

Conclusion

This was a very basic compilation of AWS offerings on the field of generative AI. It's certainly missing a lot of information and please let me know if there's inaccurate information in it.

From the small content I was able to compile, it's clear the AWS focus on providing flexible solutions that can be "plugged" into a company's data source to increase the usefulness of AI-based tools.

Apart from checking the offerings from competing providers such as Azure and GCP, I hope I can try some of those features myself since I haven't done so yet.

References

You get articles that match your needs
You can efficiently read back useful information
You can use dark theme

What you can do with signing up