Python Bytes

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Brian #1: rich-codex by Phil Ewels suggested by Will McGugan “A GitHub Action / command-line tool which generates screen grab images of a terminal window, containing command outputs or code snippets.” Generate images from commands embedded in markdown files, like README.md, for example. Searches through markdown files for stuff like:  then runs the command, and generates the image. Can be done within a GitHub action Can also send code snippets or json to rich-cli, then generate an image. You can also have commands in a config file, Very easy to use, makes very professional looking images for documentation, that’s always up to date. Michael #2: Pydastic via Roman Right, by Rami Awar Pydastic is an elasticsearch python ORM based on Pydantic. Core Features Simple CRUD operations supported Sessions for simplifying bulk operations (a la SQLAlchemy) Dynamic

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Michael #1: Python License tracker by Tom Nijhof/Nyhof Every package depends on other package with sometimes different licenses. Tom made a tool to find out what licenses you all need for a project: PyTest alone needs 4 different licenses for itself and its dependencies. Tensorflow is even worst Brian #2: undataclass Trey Hunner As a teaching aid, and to show how much dataclasses do for you, this is a module and an application that converts dataclasses to normal classes, and fills in all of the dunder methods you need. Example in app: from dataclasses import dataclass @dataclass() class Point: x: float y: float z: float Converts to class Point: __match_args__ = ('x', 'y', 'z') def __init__(self, x: float, y: float, z: float) -> None: self.x = x

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Special guest: Nick Muoh Brian #1: picologging From a tweet by Anthony Shaw From README.md “early-alpha” stage project with some incomplete features. (cool to be so up front about that) “Picologging is a high-performance logging library for Python. picologging is 4-10x faster than the logging module in the standard library.” “Picologging is designed to be used as a drop-in replacement for applications which already use logging, and supports the same API as the logging module.” Now you’ve definitely got my attention. For many common use cases, it’s just way faster. Sounds great, why not use it? A few limitations listed: process and thread name not captured. Some logging globals not observed: logging.logThreads, logging.logMultiprocessing, logging.logProcesses Logger will always default to the Sys.stderr and not observe

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Special guest: Gina Häußge, creator & maintainer of OctoPrint Michael #1: beanita Local MongoDB-like database prepared to work with Beanie ODM So, you know Beanie - Pydantic + async + MongoDB And you know Mongita - Mongita is to MongoDB as SQLite is to SQL Beanita lets you use Beanie, but against Mongita rather than a server-based MongoDB server Brian #2: The Good Research Code Handbook Patrick J Mineault “for grad students, postdocs and PIs (principle investigator) who do a lot of programming as part of their research.” lessons setup git, virtual environments, project layout, packaging, cookie cutter style style guides, keeping things clean coding separating concerns, separating pure functions and those with side effects, pythonic-ness testing unit testing, testing with side effects, … (incorrect definition

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: Polars: Lightning-fast DataFrame library for Rust and Python Suggested by a several listeners “Polars is a blazingly fast DataFrames library implemented in Rust using Apache Arrow Columnar Format as memory model. Lazy | eager execution Multi-threaded SIMD (Single Instruction/Multiple Data) Query optimization Powerful expression API Rust | Python | ...” Python API syntax set up to allow parallel and execution while sidestepping GIL issues, for both lazy and eager use cases. From the docs: Do not kill parallelization The syntax is very functional and pipeline-esque: import polars as pl q = ( pl.scan_csv("iris.csv") .filter(pl.col("sepal_length") > 5) .groupby("species") .agg(pl.all().sum()) ) df = q.collect() Polars User Guide is excellent and looks like it’s entirely

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Michael #1: auto-py-to-exe Converts .py to .exe using a simple graphical interface A good candidate to install via pipx For me, just point it at the top level app.py file and click go Can add icons, etc. Got a .app version and CLI version (I think

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: The Python GIL: Past, Present, and Future Bary Warsaw and Paweł Polewicz Michael #2: Announcing the PyOxy Python Runner PyOxy is all of the following: An executable program used for running Python interpreters. A single file and highly portable (C)Python distribution. An alternative python driver providing more control over the interpreter than what python itself provides. A way to make some of PyOxidizer's technology more broadly available without using PyOxidizer. PyOxidizer is often used to generate binaries embedding a Python interpreter and a custom Python application. However, its configuration files support additional functionality, such as the ability to produce Windows MSI installers, macOS application bundles, and more. The pyoxy executable also embeds a copy of the Python standard library and import

Watch the live stream: Watch on YouTube About the show Sponsored: RedHat: Compiler Podcast Special guests Mark Little Ben Cosby Michael #1: libgravatar A library that provides a Python 3 interface to the Gravatar APIs. If you have users and want to show some sort of an image, Gravatar is OK PyPI uses this for example (gravatar, not necessarily this lib) Usage: >>> g = Gravatar('myemailaddress@example.com') >>> g.get_image() 'https://www.gravatar.com/avatar/0bc83cb571cd1c50ba6f3e8a78ef1346' Brian #2: JSON to Pydantic Converter Suggested by Chun Ly, “this awesome JSON to @samuel_colvin's pydantic is so useful. It literally saved me days of work with a complex nested JSON schema.“ “JSON to Pydantic is a tool that lets you convert JSON objects into Pydantic models.” It’s a live site, where you can plop JSON on one the left, and Pydantic models show up on the right. There’s a couple options: Specify every field as Optional Alias camelCase fields as snake_case It’s

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1:distinctipy “distinctipy is a lightweight python package providing functions to generate colours that are visually distinct from one another.” Small, focused tool, but really cool. Say you need to plot a dynamic number of lines. Why not let distinctipy pick colors for you that will be distinct? Also can display the color swatches. Some example palettes here: https://github.com/alan-turing-institute/distinctipy/tree/main/examples from distinctipy import distinctipy # number of colours to generate N = 36 # generate N visually distinct colours colors = distinctipy.get_colors(N) # display the colours distinctipy.color_swatch(colors) Michael #2: Soda SQL Soda SQL is a free, open-source command-line tool. It utilizes user-defined input to prepare SQL queries that run tests on dataset in a data source to find invali

Watch the live stream: Watch on YouTube About the show Sponsored: RedHat: Compiler Podcast Special guest: Tonya Sims Michael #1: Pathy: a Path interface for local and cloud bucket storage via Spencer Pathy is a python package (with type annotations) for working with Cloud Bucket storage providers using a pathlib interface. It provides an easy-to-use API bundled with a CLI app for basic file operations between local files and remote buckets. It enables a smooth developer experience by letting developers work against the local file system during development and only switch over to live APIs for deployment. Also has optional local file caching. From Spenser The really cool function is "Pathy.fluid" which can take any type of local, GCS, or S3 path string and then just give you back a Path object that you can interact with agnostic of what platform it was. So this has worked amazingly for me in local testing since i can just change the file path from the "s3://bucket/path" that i use in prod to a local "

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: pyscript Python in the browser, from Anaconda. repo here Announced at PyConUS “During a keynote speech at PyCon US 2022, Anaconda’s CEO Peter Wang unveiled quite a surprising project — PyScript. It is a JavaScript framework that allows users to create Python applications in the browser using a mix of Python and standard HTML. The project’s ultimate goal is to allow a much wider audience (for example, front-end developers) to benefit from the power of Python and its various libraries (statistical, ML/DL, etc.).” from a nice article on it, PyScript — unleash the power of Python in your browser PyScript is built on Pyodide, which is a port of CPython based on WebAssembly. Demos are cool. Note included in README: “This is an extremely experimental project, so expect things to break!” Michael #2: Memray from Bloomberg

Watch the live stream: Watch on YouTube About the show Sponsored: RedHat: Compiler Podcast Special guest: Anna Astori Michael #1: Take Your Github Repository To The Next Level

Watch the live stream: Watch on YouTube About the show Sponsored by Mergify! Special guest: Pat Decker Michael #0: New live stream / recording time: 12pm US PT on Tuesdays. Please subscribe to our YouTube channel to get notified and be part of the episodes. Brian #1: BTW, don’t make a public repo private How we lost 54k GitHub stars Jakub Roztočil HTTPie kinda sorta accidentally flipped their main repo to private for a sec. And dropped the star count from 54k to 0 oops They’re back up to 16k, as of today. But ouch. “HTTPie is a command-line HTTP client. Its goal is to make CLI interaction with web services as human-friendly as possible. HTTPie is designed for testing, debugging, and generally interacting with APIs & HTTP servers. The http & https commands allow for creating and sending arbitrary HTTP requests. They use simple and natural syntax and provide formatted and colorized output.” Actually, pretty cool tool to use for developing and testing APIs. Michael #2: The counter-intuitive

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Brian Skinn (Twitter | Github) Michael #1: OpenBB wants to be an open source challenger to Bloomberg Terminal OpenBB Terminal provides a modern Python-based integrated environment for investment research, that allows an average joe retail trader to leverage state-of-the-art Data Science and Machine Learning technologies. As a modern Python-based environment, OpenBBTerminal opens access to numerous Python data libraries in Data Science (Pandas, Numpy, Scipy, Jupyter) Machine Learning (Pytorch, Tensorflow, Sklearn, Flair) Data Acquisition (Beautiful Soup, and numerous third-party APIs) They have a discord community too BTW, seem to be a successful open source project: OpenBB Raises $8.5M in Seed Round Funding Following Open Source Project Gamestonk Terminal's Success Great graphics / gallery here. Way more affordable than the $1,900/mo/user for the Bloomberg Terminal Brian #2: Python f

Watch the live stream: Watch on YouTube About the show Sponsored by: Microsoft for Startups Founders Hub. Special guest: Vuyisile Ndlovu Brian #1: dunk - a prettier git diff Darren Burns Uses Rich “⚠️ This project is very early stages” - whatever, I like it. Recommendation is to use less as a pager for it git diff | dunk | less -R Michael #2: Is your Python code vulnerable to log injection? via Adam Parkin Let’s just appreciate log4jmemes.com for a moment Ok, now we can talk about Python We can freak our the logging with line injection "hello'.\nINFO:__main__:user 'alice' commented: 'I like pineapple pizza" Results in two lines for one statement INFO:__main__:user 'bob' commented: 'hello'. INFO:__main__:user 'alice' commented: 'I like pineapple pizza'. The safest solution is to simply not log untrusted text. If you need to store it for an audit trail, use a database. Alternatively, structured logging can prevent newline-based attacks. Padding a ton? One such case is abusing

Watch the live stream: Watch on YouTube About the show Sponsored by: Microsoft for Startups Founders Hub. Special guest: Thomas Gaigher, creator/maintainer pypyr taskrunner Michael #1: March Package Madness via Chris May Start with 16 packages They battle it out 2-on-2 in elimination rounds Voting is once a week So go vote! Brian #2: nbpreview “A terminal viewer for Jupyter notebooks. It’s like cat for ipynb files.” Some cool features pretty colors by default piping strips formatting, so you can pass it to grep or other post processing automatic paging syntax highlighting line numbers and wrapping work nicely markdown rendering images converted to block, character, or dots (braille) dataframe rendering clickable links Thomas #3: pyfakefs A fake file system! It intercepts all calls that involve the filesystem in Python - e.g open(), shutil, or pathlib.Path. This is completely transparent - your functional code does not know or need to know that under the hood it's been disconnected from

Watch the live stream: Watch on YouTube About the show Sponsored by FusionAuth: pythonbytes.fm/fusionauth Special guest: Ian Hellen Brian #1: gensim.parsing.preprocessing Problem I’m working on Turn a blog title into a possible url example: “Twisted and Testing Event Driven / Asynchronous Applications - Glyph” would like, perhaps: “twisted-testing-event-driven-asynchrounous-applications” Sub-problem: remove stop words ← this is the hard part I started with an article called Removing Stop Words from Strings in Python It covered how to do this with NLTK, Gensim, and SpaCy I was most successful with remove_stopwords() from Gensim from gensim.parsing.preprocessing import remove_stopwords It’s part of a gensim.parsing.preprocessing package I wonder what’s all in there? a treasure trove gensim.parsing.preprocessing.preprocess_string is one this function applies filters to a string, with the defaults almost being just what I want: strip_tags() strip_punctuation() strip_multiple_whitespaces() str

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Emily Morehouse-Valcarcel Michael #1: Async and await with subprocesses by Fredrik Averpil People know I do all sorts of stuff with async Lots of cool async things are not necessarily built into Python, but our instead third-party packages E.g. files via aiofiles But asyncio has asyncio.create_subprocess_exec Fredrik’s article has some nice examples I started using this for mp3 uploads and behind the scenes processing for us Brian #2: Typesplainer Arian Mollik Wasi, @wasi_master Suggested by Will McGugan Now released a vscode extension for that! Available on vscode as typesplainer Emily #3: Ibis Project via Marlene Mhangami “Productivity-centric Python data analysis framework for SQL engines and Hadoop” focused on: Type safety Expressiveness Composability Familiarity Marlene wrote an excellent blog post as an introduction Works with tons of different backends, either dir

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Anne Barela Brian #1: The Adam Test : 12 Questions for New Dependencies Found through a discussion with Ryan Cheley, who will be on an upcoming episode of Test & Code, talking about Managing Software Teams. The Joel Test dates back to 2000, and some of it is a bit dated. I should probably do a Test & Code episode or pythontest article on my opinions of this at some point. Nice shameless plugs, don’t you think? The Joel Test is 12 questions and is a “highly irresponsible, sloppy test to rate the quality of a software team.” “The Adam Test” is 12 questions “to decide whether a new package we’re considering depending on is well-maintained.” He’s calling it “The Well-Maintained Test”, but I like “The Adam Test” Here’s the test: Is it described as “production ready”? Is there sufficient documentation? Is there a changelog? Is someone responding to bug reports? Are there suf

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: Physics Breakthrough as AI Successfully Controls Plasma in Nuclear Fusion Experiment Interesting break through using AI Is Python at the center of it? With enough digging, the anwswer is yes, and we love it! Brian #2: PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library Accepted for Python 3.11 This PEP proposes basing the standard library support for reading TOML on the third-party library tomli Michael #3: Thread local threading.local: A class that represents thread-local data. Thread-local data are data whose values are thread specific. Just create an instance of local (or a subclass) and store attributes on it You can even subclass it. Brian #4: What is a generator function? Trey Hunner Super handy, and way easier than you think, if you’ve never written your own. Really, it’s just a function that uses yield instead of return and supplies one element at a time