Measuring the CPU and Memory Usage for Python subprocess

The article talks about a method for measuring the resources used by a process invoked by subprocess.run or subprocess.Popen. We can use psutil to get realtime usage data of a process. Using psutil may not provide accurate resource measurements for short-lived processes, as it samples usage at intervals and can miss brief spikes. We want a method to get the final resource usage of a process after it finishes. The method leverages multiprocessing. Continue reading

tac -- the ignored reverse of cat

When grep a very large log file to find the last occurrence of some string, it is much than efficient to back iterate the files from the last line than methods using tail or tail -n. And I just found tac from coreutils, best suiting the job. tac is the reverse of cat, the job I mentioned above can be done via: tac very-large-log-file.log|grep -m 1 some-string This is very useful. Continue reading

Python version of Rust Result Sum Type

I try to implement a Python version of the Result<T, E> sum type in rust which can be used with match/case pattern matching. The pattern matching with sub-patterns like case Err(e) could be implemented in Python with the __match_args__ dunder attribute. This is how Result[T, E], OK() and Err() defined: import typing T = typing.TypeVar('T') E = typing.TypeVar('E') class Result[T, E]: __match_args__ = ('_val',) def __init__(self, val): self._val = val class Ok[T](Result[T, . Continue reading

Run wasm built with wasm-pack with pythonmonkey

I have been trying to run wasm code built with wasm-pack with pythonmonkey, a Mozilla SpiderMonkey JavaScript engine embedded into the Python Runtime. Consider a lib.rs as follow: use wasm_bindgen::prelude::*; #[wasm_bindgen] pub fn add(a: i32, b: i32) -> i32 { return a + b; } With wasm-pack, we can build the src code to wasm wasm-pack build --target nodejs --no-typescript We will get a .wasm file and a .js wrapper file. Continue reading

Combine mesop with natural abilities of flask

mesop is popular with AI apps, since is can be used to build web apps with Python without frontend works. mesop is developed upon the popular frontend framework flask. How about combine the natural abilities of flask while developing with mesop? Let do a little research. Access flask request instance(including headers, cookies) In any mesop page, you may simply access flask.request, flask.request.cookies, flask.request.headers by importing flask. For example import flask import mesop as me @me. Continue reading

Create a partial model definition of a pydantic model

Sometimes I want to create a partial model definition of a pydantic model. For example, if I want to expose a database model from API, some of the fields I would like to keep in secret, or I just want to expose very few fields in a summary API. I don’t want to write model definition twice. I would like to write something like ModelExposed = partial_model(ModelSource, 'ModelExposed', include=["title", "description"]) So I write a funciton as below to do it, including support of passing list of excluding fields, copying computed fields and validators. Continue reading

psql vs. mysql shell command cheatsheet

MySQL Postgres list databases SHOW DATABASES \l change to a database USE <db_name> \c <db_name> list tables SHOW TABLES \dt describe a table DESC <table_name> \d <table_name> show the create table sql SHOW CREATE TABLE <table_name> pg_dump -st <table_name> <db_name> explain a query EXPLAIN <sql_statement> EXPLAIN <sql_statement> expanded(vertical) display \G \gxor use \x to switch get help ? <SQL STATEMENT>HELP <SQL STATEMENT> \h <SQL STATEMENT> get active processes SHOW [FULL] PROCESSLIST; SELECT pid, usename, state, query, query_start, application_name FROM pg_stat_activity WHERE state = ‘active’; kill a process KILL <pid>; SELECT pg_terminate_backend(<pid>); quit shell QUIT \q