Subscribe
Hands-on with horizontally scaling Python for production
By MAJ Brent Stone, Cyber Solutions Development, U.S. Army Cyber Command
This was a hands-on Python programming workshop. Python can be challenging to use in production when "real-world" workloads involving megabits per second (Mbps) of streaming data or terabytes of stored data are involved. The Global Interpreter Lock (GIL) means that a Python interpreter is effectively single-threaded and can't take advantage of modern processors' capacity for parallel computation. Laterally scaling workloads across many Python interpreters is one of the most viable workarounds to the shortcomings of the GIL. This workshop will introduce you to two leading frameworks for doing this: Celery and Dask (via Prefect). This workshop will walk through establishing an Extract, Transform, Load (ETL) pipeline in each framework which reads and writes from Redis and a MinIO locally hosted S3 bucket.
| Date Taken: | 02.28.2024 |
| Date Posted: | 03.01.2024 16:49 |
| Photo ID: | 8265421 |
| VIRIN: | 240228-O-PX639-8475 |
| Resolution: | 6720x4480 |
| Size: | 7.89 MB |
| Location: | AUGUSTA, GEORGIA, US |
| Web Views: | 19 |
| Downloads: | 2 |
This work, AvengerCon VIII - Hands-on with horizontally scaling Python for production workshop [Image 17 of 17], by Steven Stover, identified by DVIDS, must comply with the restrictions shown on https://www.dvidshub.net/about/copyright.
No keywords found.
