Though AWS Lambda supports languages like C# and Go, many developers choose to use Python with it due to its ease of use and flexibility.
Plus, AWS Lambda gives you the flexibility to integrate your Python code easily with other services such as API Gateway S3 and DynamoDB. This makes Python an excellent choice for building efficient and multifaceted serverless apps.
AWS Lambda supports Python 3.6 through 3.9, and in April 2023, Amazon Web Services announced in a blog post that it had expanded support to Python 3.10.
So, Python developers can now take advantage of all the new features and improvements that Python 3.10 brings to the table when developing applications on AWS Lambda.
This post will explore the improvements and features introduced in Python 3.10 and discuss a few ways you can use Python 3.10 with AWS Lambda.
What is AWS Lambda?
Lambda is Amazon's serverless computing platform that enables you to run code as a response to a specific event(s). For example, the event may be an HTTP request made to an API Gateway or some change in the data in an S3 bucket.
So, put simply, AWS Lambda enables the execution of code without the need to manage any type of infrastructure.
Additionally, AWS Lambda removes the need for developers and businesses to pay for servers to run their code on. With Lambda, the user need only pay for the time their code needs to execute.
This makes using Lambda a viable option for many use cases, including but not limited to real-time processing of data from IoT devices and machine learning.
Python 3.10: New Features
One of the new features in Python 3.10 is structural pattern matching, allowing developers to match patterns against data structures. This Python Enhancement Proposal (PEP, specifically PEP 634) makes processing complex data structures easier.
Another new feature is PEP 604, replacing the syntax for writing union types. As opposed to the Union[X, Y], developers can now simply use X | Y to write union types.
This simplification of the syntax makes for easier writing of union types, makes the code more readable, and helps reduce boilerplate code.
Parenthesized context managers were also introduced in this new version of Python, allowing developers to manage resources such as database connections and file handles more easily. Besides ensuring that the resources are released correctly, this new feature allows developers to use the "with" statement easily.
The new PEP 647 allows the definition of custom-type guards that can handle custom data types. What's interesting is that this addition also allows developers to refine the built-in types.
Not to mention, the feature makes the code easier to read and maintain, which is especially significant in code involving custom data types and complex data structures.
Lastly, Python 3.10 introduces improved error messages that give developers more information about the cause of the error. The new error messages also suggest possible solutions to resolving the error.
Python 3.10 Performance Improvements
Python 3.10 introduces several performance improvements that make function calls, attribute lookups, and data type construction faster and more efficient.
With these improvements, you can expect a significant reduction in CPU time and memory usage, especially if you use functions with complex annotations or several parameters.
Another notable performance improvement in Python 3.10 is delivered as PEP 590, representing the enhanced vectorcall calling convention. This addition speeds up Python function calls that involve multiple arguments.
Several built-in functions, including map(), bool(), and filter(), see a 1.26x performance improvement as a result of PEP 590.
Furthermore, since PEP 590 delays the creation of the annotation dictionary until it is required, your programs run faster. This is because the overhead of initializing the dictionary is avoided when the function is defined.
Moreover, Python 3.10 comes with the "per opcode cache," a new feature that improves the LOAD_ATTR instruction, which loads the attributes from objects.
This new feature caches frequently accessed attributes specific to each LOAD_ATTR instruction, resulting in a 36% speedup for regular attributes and a 44% speedup for attributes defined using the slots mechanism.
Further performance testing has revealed that data type constructors such as bytes() and bytearray() are between 30% and 40% faster for small objects.
Lambda functions that read and process Gzip compressed files can benefit from a new mechanism called _BlocksOutputBuffer, which prevents excessive memory footprint and speeds up decompression.
As a result, bz2 decompression is now 1.09x faster, lzma decompression is 1.20x faster, and GzipFile read is 1.11x faster in Python 3.10.
It's best to upgrade to Python 3.10 soon to take advantage of these improvements. Check out the official release notes to find more details about the improvements.
AWS Lambda Now Supports Python 3.10
AWS Lambda now supports Python 3.10 as a container base image and managed runtime.
This update offers developers creating serverless applications in Lambda and uses the entire range of the new Python version's enhancements, including:
- Pattern matching for data structures,
- Better error handling, and
- Parenthesized context managers that make managing resources easier.
You can upload the code through the Lambda console and select the Python 3.10 runtime to deploy Lambda functions.
Of course, you can also use the AWS Serverless Application Model, AWS CloudFormation, or the AWS CLI to deploy serverless applications.
What's more, the Python 3.10 base image that is available in AWS allows you to deploy Python 3.10 functions as a container image.
This makes it important to migrate existing Lambda functions running on previous Node.js versions only after ensuring they are compatible with Node.js 18. Do not update your runtime to Python 3.10 before doing this.
When you do update your runtime, AWS will automatically apply the updates to the Python-managed and AWS-managed runtime.
Note that the new runtime is only available in the regions where AWS Lambda is available. Meaning you cannot access Python 3.10 in the GovCloud Regions and in China.
If you're looking for more details about AWS Lambda, it's best to look at its official page. Also, note that the old deployment size limit of 250 MB is still the same.
So, if you're using a lot of third-party libraries, you may need to consider using a package manager, such as pip, to manage your dependencies and keep your package size under the limit.
Another important consideration is cold starts. When a Lambda function is invoked for the first time, AWS needs to provision a new container to run the function.
Using Python 3.10 in Lambda
There are many approaches to building and deploying functions using Python 3.10, and they include:
- AWS CDK
- AWS CLI
- AWS Management Console
- AWS SAM
- AWS SDK
- Infrastructure as Code (IaC)
You always have the option to rely on the Python 3.10 container base image that AWS supplies.
To use Python 3.10 runtime with the AWS Management Console, navigate to the "Create function" page and click on the "Runtime" dropdown. Here, you can set the version to Python 3.10.
It is also possible to update existing functions to the new Python version. You can do this by finding the function you want to update and hitting the "Edit" button in the Runtime settings panel. You can set the Python version in the dropdown you will find here.
On the other hand, if you want to use Python 3.10 with an AWS Lambda Container image, you must change Python's base image version by altering the FROM statement in the Dockerfile.
You can read about this method and other methods of using Python in AWS Lambda in detail in the company's blog post.