If haven't searched both "python" and "robots.txt" in the same input box, I would not ever know that Python Standard Library could parse robots.txt with urllib.robotparser. But the official document of urllib.robotparser doesn't go into detail. With the document, you could check whether a url can be fetch with a robot …

Read More

Pipelining streamed data events directly into Amazon S3 via the AWS Kinesis Firehose service is convenient and efficient. And we could use Python with boto3 to consume the data directly from S3. This allows for seamless storage of your data, ensuring its integration and accessibility. Mostly, we are dealing with …

Read More

Here in the post, I will share some examples about the contextvars (new in Python 3.7) and thread local. Default Value In the module level, use ContextVar.set or directly setattr for a thread local variable, won't successfully set a default value, the value set won't take effect in another thread. To ensure a default …

Read More