Learning source: Chai aur Code — Python Video 3

In this video, I started exploring what happens behind the scenes when we run a Python program. What began with a simple example of importing a function from one file into another led me to some interesting concepts: bytecode, __pycache__, the Python Virtual Machine (PVM), and different Python implementations.

I'll be honest: I got confused, especially when trying to understand what the PVM actually does and how Python's rules differ from its implementation.

So, let's break down what I understood.

1. Starting With a Simple Python Program

We started with an example involving two Python files: chai.py and hellochai.py:

chai.py

def hello():
    print("Hello from Chai aur Code!")

Now, suppose we want to use this function in another file.

hellochai.py

from chai import hello

hello()

Here, we're importing the hello() function from chai.py and calling it in hellochai.py.

We can run the second file using the terminal command:

python hellochai.py

Python executes the program, and we get the output:

Hello from Chai aur Code!

But here's where things get interesting.

When we ran the program, a folder named __pycache__ appeared, containing a .pyc file.

Why did Python create these files? What are they for? And what do they have to do with bytecode?

To understand that, we first need to understand what happens when Python executes our code.

2. What Is Bytecode?

When we write Python code, we write it in a human-readable form.

For example:

print("Hello")

The computer doesn't execute this source code directly as native machine instructions. In the standard Python implementation, CPython, the source code is compiled into an intermediate form called bytecode.

Bytecode is a lower-level representation of our Python program. It consists of instructions that the Python runtime can execute.

Think of it as a step between the code we write and the actual execution of our program.

Bytecode is not the same as machine code. Machine code consists of instructions that a processor can execute directly, whereas Python bytecode is designed for a Python virtual machine.

One thing worth remembering is that Python bytecode is not universally compatible across every Python implementation and version. Its format and instructions can differ.

Why does Python use bytecode?

Bytecode provides a representation that the Python runtime can execute without having to interpret the original source text from scratch for every instruction.

When CPython runs a program, the process broadly involves:

1. Reading the Python source code.

2. Compiling it into bytecode.

3. Executing that bytecode using the Python runtime.

The compilation and execution happen as part of running the program, so we usually don't need to perform these steps manually.

However, there is an important distinction: compiling code into bytecode does not automatically mean that a .pyc file will be saved.

That brings us to the __pycache__ folder.

3. What Are __pycache__ and .pyc Files?

When I ran the program that imported a function from another file, I noticed that Python automatically created a __pycache__ folder.

Inside it was a file ending in .pyc.

So, what exactly are these?

A .pyc file contains cached compiled Python code, usually bytecode produced by CPython.

When a module such as chai.py is imported, CPython can compile it and cache the resulting bytecode in a .pyc file. The next time the module is imported, Python may reuse that cached version if it is still valid.

This can save time because Python may not need to compile the module from its source code again.

Why is the folder called __pycache__?

The __pycache__ folder provides a place to store cached bytecode files instead of mixing them with our regular source files.

For example, our project might look like this:

my_project/
├── chai.py
├── hellochai.py
└── __pycache__/
    └── chai.cpython-313.pyc

The exact filename depends on the Python implementation and version. The example above is illustrative.

Keeping these generated files in a separate folder helps keep our project organized.

Does Python create a .pyc file every time we run a program?

Not necessarily.

In our example, chai.py is imported by hellochai.py, so CPython can cache the compiled bytecode for chai.py.

However, when we execute a regular Python script directly, Python doesn't normally save a .pyc file for that main script simply because we ran it.

The script is still compiled to bytecode for execution. The difference is that the compiled bytecode is not necessarily written to a cache file.

Also, .pyc files don't necessarily get regenerated every time we run a program. Python checks whether cached bytecode is usable and recompiles the source when needed.

One more clarification: .pyc files are not usually called frozen binaries. They are cached compiled Python code. Frozen modules and frozen bytecode are related but distinct concepts.

4. What Exactly Is the Python Virtual Machine (PVM)?

This was the part that confused me the most.

I understood that Python code gets compiled into bytecode, but then I wondered:

If bytecode is generated, what does the PVM actually do?

The PVM is commonly described as the component that executes Python bytecode.

In CPython, the runtime contains an interpreter that processes bytecode instructions and carries out the operations they describe.

For example, when we write:

a = 10
b = 20
print(a + b)

The runtime needs to perform several operations: create and assign objects, retrieve values, perform addition, and display the result.

The bytecode represents the operations to be performed, and the interpreter executes the corresponding instructions.

Does the PVM come with Python?

When we install a Python implementation such as CPython, we get the runtime and interpreter components needed to execute Python programs.

We don't normally install a separate PVM ourselves.

Also, PVM is a conceptual term used to explain Python's execution model. The precise implementation details depend on which Python implementation we're using.

Does Python execute source code directly if there are no imports?

This was another point I needed to clarify.

No. In CPython, the absence of imports does not mean the source code bypasses bytecode.

Even a simple script such as:

print("Hello, Python!")

is compiled into bytecode for execution.

The important distinction is between compiling code for execution and saving compiled bytecode to a .pyc file.

A script can be compiled into bytecode and executed without saving a .pyc file for that script.

5. How Does the PVM Execute Bytecode?

One way to understand the process is to imagine the interpreter working through a sequence of instructions.

Broadly, it follows a cycle:

Fetch an instruction → Execute the instruction → Fetch the next instruction → Execute the next instruction → Repeat

This continues as the program runs.

The interpreter processes instructions and performs the corresponding operations. Depending on the instruction, this could involve arithmetic, calling a function, accessing an object, or controlling the program's flow.

The PVM isn't simply a folder containing bytecode, nor is it the same thing as the .pyc file. The .pyc file stores cached compiled code, while the interpreter is responsible for executing the instructions.

Why is it called a runtime engine?

A runtime is the environment that supports a program while it executes.

In Python, the runtime does more than process bytecode instructions. It also supports important operations such as:

• Memory management: Managing memory used by Python objects.

• Object management: Handling Python objects and their behavior.

• Execution state: Keeping track of the program's current execution.

• Function calls: Managing how functions are called and how execution moves between them.

These responsibilities are part of the broader Python runtime, rather than all being performed by one single component called the PVM.

Thinking of the runtime as the machinery that helps a program execute made the concept easier for me to understand.

6. What Is a Python Implementation?

Towards the end of the video, we were introduced to different Python implementations.

At first, I was confused by the word implementation. I wondered how it could be different from Python's rules.

A simple way to understand the distinction is this:

Python's language specification describes how Python code should behave. An implementation is the actual software that makes that behavior happen.

The language defines things such as syntax, expressions, data types, and expected behavior. An implementation provides the machinery needed to execute programs according to those rules.

Different implementations can follow the same language specification while using different technologies internally.

Let's look at some examples.

A. CPython

CPython is the standard and most widely used implementation of Python.

It is primarily written in C, and it compiles Python source code into bytecode that its interpreter executes.

When we download Python from the official Python website for a typical installation, we're generally using CPython.

B. PyPy

PyPy is another Python implementation that focuses on execution speed.

One of its important features is a Just-In-Time (JIT) compiler, which can compile frequently executed code into machine code while the program is running.

This can make some programs run faster than they do under CPython, although the actual performance depends on the workload.

C. Jython

Jython is a Python implementation that runs on the Java Virtual Machine (JVM).

It allows Python code to work with Java classes and libraries.

The JVM is the runtime environment that executes Java bytecode and provides services for Java-based applications.

Jython is a distinct implementation, and its Python language-version support differs from modern CPython.

D. Stackless Python

Stackless Python is based on CPython and focuses on alternative ways of managing execution, including task scheduling and lightweight tasklets.

It aims to reduce dependence on the traditional C call stack used in ordinary function calls and supports alternative approaches to concurrency.

In simpler terms, it explores different ways to manage tasks and execution rather than relying entirely on the usual call-stack model.

E. IronPython

IronPython is a Python implementation designed to work with the .NET ecosystem.

It allows Python code to interact with .NET libraries and technologies.

But what exactly is .NET?

.NET is a development platform from Microsoft that provides tools, libraries, and a runtime for building different types of applications.

IronPython makes it possible to use Python in that ecosystem.

A quick comparison

These implementations demonstrate that Python is not just one particular executable program. It is a programming language that can be implemented in different ways.

7. My Key Takeaways

Here's what I took away from this video:

1. Bytecode is an intermediate representation of Python code that the runtime can execute. It is not machine code.

2. __pycache__ is a folder where Python can store cached compiled bytecode for imported modules.

3. .pyc files contain cached compiled Python code and may be reused when the cache is valid.

4. The PVM is commonly used to explain how Python bytecode is executed by the interpreter.

5. The Python runtime provides the machinery required to execute and manage a running program.

6. A Python implementation is the actual software that implements the language's rules.

7. CPython, PyPy, Jython, Stackless Python, and IronPython are examples of different implementations, each with its own approach or focus.

Final Thoughts

This video made me realize that running a Python program involves more than simply giving instructions to the computer.

There's source code, bytecode, an interpreter, cached compiled files, and a runtime environment working together to make execution possible.

The concept that took me the longest to understand was the PVM. Initially, I thought bytecode was something generated only when imports happened. Now I understand that compilation for execution and caching compiled code are two different things.

I also learned that Python's language rules and its implementation aren't the same. The rules describe how the language should behave, while an implementation provides the software that makes it work.

I'm still learning these concepts, but connecting them to an actual program made them much easier to explore.

One question leads to another, and that's exactly what makes this journey interesting.

This is part of my Python learning journey, where I document what I understand from each Chai aur Code lecture, the questions I encounter, and the concepts I explore along the way.

Have you explored how Python works behind the scenes? If you have a simpler explanation, a useful resource, or a correction to something I've misunderstood, I'd love to learn from you in the comments.

Learning source: Watch Video 2 — Chai aur Code

Before coding our first program in Python, we must ask ourselves some more fundamental questions: how can we install Python, and where can we execute our code?

In this video, I learned how to install Python on our computer, explored online alternatives, and got introduced to tools like Google Colab, Anaconda, Miniconda, and conda.

Let's break down what I understood.

1. Installing Python + VS Code

The recommended approach is to install Python on our computer using the official Python website.

Installing Python locally gives us more control over our development environment. We can work with files on our computer, install packages, and build projects without depending entirely on an online coding platform.

On Windows, the installer is generally a .exe file. Windows also provides the Python launcher (py) in many installations.

On macOS, the official installer is generally a .pkg file, and we commonly use the python3 command to run Python 3.

For example, depending on the operating system and installation, we might run Python using:

python --version

or:

python3 --version

The command that works depends on how Python is installed and configured.

Although online environments are convenient, installing Python locally helps us work in a more complete development environment and avoid certain limitations we might encounter online. However, a local installation can also have setup issues, so it is important to understand how our environment works.

2. What If I Don't Want to Install Python?

If we don't want to install Python immediately, there are other tools that allow us to write and run code through a browser.

Let's look at the alternatives I learned about.

A. OnlineGDB

Try OnlineGDB

OnlineGDB is a browser-based coding environment that allows us to write, execute, and debug programs in several programming languages, including Python.

We can use it to practise coding without setting up a complete local development environment.

It is useful when we want to:

• Try small code examples.

• Practise programming exercises.

• Test our understanding of a concept quickly.

• Debug simple programs.

OnlineGDB is an example of an online IDE (Integrated Development Environment). An IDE brings together tools that help us write, run, and debug code.

Online coding editors are convenient because we can start coding directly in our browser. However, they may have limitations when we move towards larger projects or need a specialized development environment.

B. Google Colab

Try Google Colab

Google Colab is another browser-based environment for writing and running Python code. However, it is particularly useful for data science, machine learning, and working with data.

One of its main advantages is that it provides a cloud-based environment with many commonly used libraries already available. This means we often don't need to spend time installing and configuring every library ourselves.

This is especially helpful when working with data analysis, graphs, visualizations, and machine learning experiments.

Since the code runs in a cloud environment, we need a reliable internet connection. The available computing resources also depend on the environment and access level.

What Is a Notebook?

While exploring Google Colab, I learned about notebooks.

A notebook is a document made up of individual sections called cells. These cells can contain different types of content, such as:

• Code

• Text and notes

• Images

• Data and tables

• Graphs and visualizations

• Outputs generated by running code

Unlike a traditional Python script, where we commonly execute the file as a whole, a notebook allows us to run individual cells and see their outputs.

For example, we could use one cell to load a dataset, another to analyse it, and another to create a graph.

This makes notebooks particularly useful for data science because we can experiment with code step by step and immediately inspect the results.

OnlineGDB vs Google Colab

Although both allow us to write Python code through a browser, their main uses are different.

Neither tool is universally better. The right choice depends on what we want to accomplish.

3. Anaconda and Miniconda

The next tools I learned about were Anaconda and Miniconda.

Both are related to Python package and environment management, but they differ in what they include when installed.

A. Anaconda

Explore Anaconda

Anaconda is a Python distribution designed mainly for data science and scientific computing.

It comes with Python, conda, and a collection of commonly used packages and tools.

These packages support tasks such as:

• Artificial intelligence and machine learning

• Data analysis and exploration

• Data visualization

• Scientific computing

• Data transformation

The main advantage is convenience. Many useful packages and tools are included, so we can start working without installing everything separately.

However, because it includes a large collection of packages, the installation requires more disk space.

B. Miniconda

Explore Miniconda

Miniconda is a smaller, minimal installer for working with Python and conda.

Instead of installing a large collection of packages upfront, it provides the essentials, including conda and a Python installation, so we can install the additional packages we need later.

This gives us more control over our environment and avoids installing a large number of packages that we might never use.

Anaconda vs Miniconda

Neither option is automatically better. Anaconda can be convenient if we want many data science tools ready to use, while Miniconda is useful if we prefer to install only what we need.

4. So, What Exactly Is Conda?

This was another concept I wanted to understand clearly.

Conda is a package and environment manager. It helps us install and manage packages, handle dependencies, and create isolated environments for different projects.

Let's understand these two main uses.

Package Management

A package is reusable software that we can use in our programs.

For example, NumPy and pandas are Python libraries that provide functionality for numerical calculations and data analysis.

Instead of implementing everything ourselves, we can install and use these libraries in our projects.

Conda helps us install packages and manage their dependencies.

Environment Management

Imagine that we're working on two different projects.

Project A requires one version of a library, while Project B requires another version. Installing everything into a single shared environment could lead to dependency conflicts.

Conda allows us to create separate environments for these projects, each with its own packages and configurations.

This helps keep projects organized and reduces the chance of dependencies interfering with one another.

One important distinction: Anaconda and Miniconda are distributions or installers, whereas conda is the tool used to manage packages and environments.

They're related, but they aren't the same thing.

5. My Key Takeaways

Here's what I took away from this video:

1. We can install Python locally to get more control over our development environment.

2. Online alternatives like OnlineGDB allow us to write and run code without setting up Python locally.

3. Google Colab is especially useful for data science because it supports notebook-based coding and provides a cloud environment with commonly used libraries.

4. A notebook contains cells that can hold code, notes, and outputs, and we can execute individual cells.

5. Anaconda includes a broad collection of packages, whereas Miniconda provides a more minimal starting point.

6. Conda helps us manage packages, dependencies, and isolated environments for Python projects.

Final Thoughts

Before this video, I thought getting started with Python was mainly about installing it and writing code. Now I understand that there are different tools and environments available, depending on what we want to do.

For someone just starting out, knowing these options is useful. We can install Python locally for general development, use an online editor for quick experiments, or explore specialized tools like Google Colab when working with data.

I still have a lot to learn, but understanding the available tools gives me a clearer starting point.

This is just the beginning of my Python learning journey!

This is part of my learning-in-public series, where I document what I understand from each Chai aur Code Python lecture.

Have any tips for someone just starting with Python? I'd love to hear your suggestions in the comments. I'm still learning, and recommendations that can make my learning process more effective are always welcome.