PythonPro #34: NumPy 2.0, Pathway for IoT, Loose Coupling with DI, and Python Summit Insights
Bite-sized actionable content, practical tutorials, and resources for Python programmers and data scientists
Welcome to a brand new issue of PythonPro!
In today’s Expert Insight we bring you an excerpt from the recently published book, Mastering Python Design Patterns - Third Edition, which discusses the benefits of loose coupling and demonstrates how to achieve loose coupling by using dependency injection.
News Highlights: NumPy 2.0 introduces a new type system and Python API, breaking backward compatibility for enhancements; Pathway, used by NATO and F1, excels in Python ETL and RAG for IoT and geospatial tasks; and Python Language Summit 2024 focuses on security workflows, calendar versioning, and more.
Here are my top 5 picks from our learning resources today:
In today’s Research Spotlight, we are covering a paper that has introduced an impressive security guard to protect against incorrect and insecure AI generated code.
Dive in, and let me know what you think about this issue in this month’s survey and get your Pack credit for the month!
Stay awesome!
Divya Anne Selvaraj
Editor-in-Chief
🐍 Python in the Tech 💻 Jungle 🌳
🗞️News
NumPy 2.0 - an evolutionary milestone: The version introduces significant changes, including a revamped type system and a cleaner Python API, which required breaking backward compatibility. Read to learn more.
Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used): Pathway is a Python-based data processing framework, designed initially for event processing in IoT and geospatial indexing for complex scenarios like freight train operations. Read to learn more.
Python 'Language Summit' 2024 - Security Workflows, Calendar Versioning, Transforms and Lightning Talks: Talks also included subjects such as enhancing Rust bindings via PyO3, formalizing the PEP prototype process, and possibly increasing Python's presence in major app stores. Read to better understand the evolving landscape of Python programming.
💼Case Studies and Experiments🔬
An Empirical Study on the Energy Usage and Performance of Pandas and Polars Data Analysis Python Libraries: Concludes that Polars, which is based on Rust, provides greater energy efficiency than Pandas, especially when managing large dataframes. Read to be able to select data analysis libraries based on the scale of data and the computational resources available.
ESPHome Server in Python: Describes the author's journey in developing an ESPHome Protocol Server in Python for better Home Assistant integration. Read to learn how to integrate custom devices into Home Assistant for improved automation.
📊Analysis
My thoughts on Python in Excel: Evaluates the integration of Python in Excel, discussing its limitations, such as the inability to use custom packages and web APIs. Read if you are wondering about the feature’s applicability for you.
Python wheel filenames have no canonical form: Highlights a peculiar oversight in Python packaging. Read to understand the nuances and potential challenges of Python wheel filename management, especially in environments requiring high data integrity or cryptographic validation.
🎓 Tutorials and Guides 🤓
Python Mappings - A Comprehensive Guide: Delves into Python's dictionary and other mapping types, exploring their characteristics, common operations, and the usage of abstract base classes. Read to gain insights on effectively using and customizing Python mappings.
Ruff - A Modern Python Linter for Error-Free and Maintainable Code: Covers installation, usage, and customization of Ruff, emphasizing its speed and ease as a replacement for other linting tools like Flake8. Read to learn how to integrate Ruff into your development workflow.
Understanding HTTP Server by implementing in Python: Using Python's socket module, this article provides a practical example of creating a server that listens for connections, receives HTTP requests, and sends back HTML content. Read to learn how to implement a basic server in Python.
Getting Started with Google Colab - A Beginner's Guide: Google Colab is a platform that allows you to write and execute Python code directly in your web browser. Read to learn how to utilize the platform for managing notebooks, and leveraging powerful computing resources for data-intensive tasks.
How to use Streamlit and Plotly to create a multi-page web application: Using Python, Streamlit, and Plotly, you will learn how to create an interactive web application that features pages for global medal counts, country-specific analysis, and a choropleth map of global medal distribution. Read to start learning.
Sending SMS through GoIP GSM gateway using HTTP API: The GoIP GSM gateway offers a robust alternative to simple GSM modems. Read to learn how GoIP can be a practical solution for integrating SMS and VoIP capabilities into your applications, especially in cost-sensitive or geographically challenging environments.
How much math can you do in 10 lines of Python: Explores the power of Python for mathematical computations. Read to learn how Python's powerful and concise syntax can efficiently handle complex mathematical operations like matrix manipulations and vector calculations in just a few lines of code.
🔑Best Practices, Advice, and Code Optimization🔏
10 cool tricks with the * operator: Delves into advanced uses of the * (splat) and ** (double splat) operators in Python, highlighting their versatility beyond basic function argument unpacking. Read to discover simpler solutions to common programming challenges.
Understanding a Python closure oddity: Examines a Python closure behavior where closures in a loop capture the same local variable, leading to different outcomes when called eagerly versus lazily. Read to avoid potential pitfalls in similar coding scenarios.
Joining Strings in Python - A "Huh" Moment: Explores a Python quirk involving the str.join() method, contrasting it with list comprehension and generator expressions for joining strings. Read to learn how how CPython's implementation details can influence performance and memory usage in unexpected ways.
🎥I've Read Over 100 Books on Python. Here are the Top 3: Reviews a range of books for those looking to get started with Python and specializing in data science or scientific programming. Watch to learn more.
Exploring Python Task Queue Libraries with Load Test: Explores Python task queue libraries—Python-RQ, ARQ, Celery, Huey, and Dramatiq—benchmarking their performance under load. Read to be able to choose the right tool for managing and processing background jobs in Python projects.
🔍Research Spotlight: An LLM Security Guard for your AI Generated Code👮
In a paper presented at EASE 2024 titled, "LLM Security Guard for Code," Kavian et al. introduced LLMSecGuard, an open source framework integrating static code analyzers with LLMs to enhance security in code generation.
Context
Developers frequently employ LLMs such as ChatGPT and Copilot for coding tasks, but these models can sometimes generate insecure code due to their training limitations. LLMSecGuard was developed to improve scrutiny and correction of potential security flaws in code suggested by LLMs.
How it Works
LLMSecGuard utilizes a RESTful architecture, implemented in Python with frameworks like Django and Flask, and uses SQLite for data persistence. This setup facilitates the integration of the following components within the framework.
Prompt Agent: This component receives prompts and manages interactions with LLMs. It reformulates prompts based on security needs, sends them to LLMs, collects the generated code, and passes this code to other components for further processing.
Security Agent: This component is responsible for identifying security vulnerabilities in the code generated by LLMs. It sends this code to static code analyzers like Semgrep and Weggli, collects the analysis results, and identifies potential security issues.
Benchmark Agent: This component tests and evaluates the security properties of different LLMs. It uses a structured set of challenges, formatted in JSON, which include specific prompts and expected outcomes. This agent assesses each LLM's response against these challenges to determine the model’s security effectiveness.
What This Means
LLMSecGuard represents a significant advancement towards safer software development practices. It can ensure that security is a built-in feature in the automated coding tools, potentially transforming how you use LLMs for generating code.
Examining the Details
LLMSecGuard offers configurable settings to suit user preferences and operational requirements. It requires at least one LLM and one code analysis engine, with users providing necessary API details for integration. The system uses the CyberSecEval benchmark which is specifically designed to evaluate the cybersecurity aspects of LLMs functioning as coding assistants. You can substitute it with alternative benchmarks if desired.
Key configurable parameters include the "termination condition," which prevents endless processing loops by limiting code iterations, and the "benchmark interval," which schedules how frequently the security benchmarks are updated to keep pace with evolving LLM capabilities. You also have the flexibility to customize which LLMs and code analysis engines are used and can specify particular challenges for the benchmarking process.
You can learn more by reading the entire paper here and access the framework here.
Take the Survey, Get a Packt Credit!
🧠 Expert insight 📚
Here’s an excerpt from “Chapter 1: Foundational Design Principles” in the book, Mastering Python Design Patterns - Third Edition by Kamon Ayeva and Sakis Kasampalis, published in May 2024.
Following the Loose Coupling principle
As software grows in complexity, the relationships between its components can become tangled, leading to a system that is hard to understand, maintain, and extend. The principle of Loose Coupling aims to mitigate this issue.
What does it mean?
Loose coupling refers to minimizing the dependencies between different parts of a program. In a loosely coupled system, components are independent and interact through well-defined interfaces, making it easier to make changes to one part without affecting others.
Benefits
Loose coupling offers several advantages:
Maintainability: With fewer dependencies, it’s easier to update or replace individual components
Extensibility: A loosely coupled system can be more easily extended with new features or components
Testability: Independent components are easier to test in isolation, improving the overall quality of your software
Techniques for loose coupling
Two primary techniques for achieving loose coupling are dependency injection and the observer pattern. Dependency injection allows a component to receive its dependencies from an external source rather than creating them, making it easier to swap or mock these dependencies. The observer pattern, on the other hand, allows an object to publish changes to its state so that other objects can react accordingly, without being tightly bound to each other.
Both techniques aim to reduce the interdependencies between components, making the system you are building more modular and easier to manage.
… now, let’s study an example to understand how to use the dependency injection technique.
An example – a message service
In Python, you can achieve loose coupling by using dependency injection. Let’s see a simple example involving a MessageService class:
First, we define the MessageService class as follows:
class MessageService:
def __init__(self, sender):
self.sender = sender
def send_message(self, message):
self.sender.send(message)
As you can see, the class will be initialized by passing a sender object to it; that object has a send method to allow sending messages.
Second, let’s define an EmailSender class:
class EmailSender:
def send(self, message):
print(f"Sending email: {message}")
Third, let’s define an SMSSender class:
class SMSSender:
def send(self, message):
print(f"Sending SMS: {message}")
Now we can instantiate MessageService using an EmailSender object and use it to send a message. We can also instantiate MessageService using an SMSSender object instead. We add code to test both actions as follows:
if __name__ == "__main__":
email_service = MessageService(EmailSender())
email_service.send_message("Hello via Email")
sms_service = MessageService(SMSSender())
sms_service.send_message("Hello via SMS")
The complete code for this example, saved in the ch01/loose_coupling.py file, is as follows:
class MessageService:
def __init__(self, sender):
self.sender = sender
def send_message(self, message: str):
self.sender.send(message)
class EmailSender:
def send(self, message: str):
print(f"Sending email: {message}")
class SMSSender:
def send(self, message: str):
print(f"Sending SMS: {message}")
if __name__ == "__main__":
email_service = MessageService(EmailSender())
email_service.send_message("Hello via Email")
sms_service = MessageService(SMSSender())
sms_service.send_message("Hello via SMS")
To test the code, run the following command:
python3.12 ch01/loose_coupling.py
You should get the following output:
Sending email: Hello via Email
Sending SMS: Hello via SMS
In this example, MessageService is loosely coupled with EmailSender and SMSSender through dependency injection. This allows you to easily switch between different sending mechanisms without modifying the MessageService class.
You can buy Mastering Python Design Patterns - Third Edition by Kamon Ayeva and Sakis Kasampalis, here. Packt library subscribers can continue reading the entire book for free here.
On a scale of 1-10, how would you rate today’s issue of PythonPro in terms of being informative, engaging, and useful?
lowest 1 2 3 4 5 6 7 8 9 10 highest
And that’s a wrap.
We have an entire range of newsletters with focused content for tech pros. Subscribe to the ones you find the most useful here. The complete PythonPro archives can be found here.
If you have any suggestions or feedback, or would like us to find you a Python learning resource on a particular subject, take the survey or leave a comment below!