# What Happens When Python Runs? Bytecode, PVM, and Python Implementations *Learning Python with Chai aur Code, one video at a time.* **Learning source:** [Chai aur Code — Python Video 3](https://www.youtube.com/watch?v=3HTKc-ZgZbg) In this video, I started exploring what happens behind the scenes when we run a Python program. What began with a simple example of importing a function from one file into another led me to some interesting concepts: bytecode, `__pycache__`, the Python Virtual Machine (PVM), and different Python implementations. I'll be honest: I got confused, especially when trying to understand what the PVM actually does and how Python's rules differ from its implementation. So, let's break down what I understood. ## 1\. Starting With a Simple Python Program We started with an example involving two Python files: `chai.py` and `hellochai.py`: **chai.py** ```python def hello(): print("Hello from Chai aur Code!") ``` Now, suppose we want to use this function in another file. **hellochai.py** ```python from chai import hello hello() ``` Here, we're importing the `hello()` function from `chai.py` and calling it in `hellochai.py`. We can run the second file using the terminal command: ```bash python hellochai.py ``` Python executes the program, and we get the output: ```text Hello from Chai aur Code! ``` But here's where things get interesting. When we ran the program, a folder named `__pycache__` appeared, containing a `.pyc` file. Why did Python create these files? What are they for? And what do they have to do with bytecode? To understand that, we first need to understand what happens when Python executes our code. ## 2\. What Is Bytecode? When we write Python code, we write it in a human-readable form. For example: ```python print("Hello") ``` The computer doesn't execute this source code directly as native machine instructions. In the standard Python implementation, CPython, the source code is compiled into an intermediate form called **bytecode**. Bytecode is a lower-level representation of our Python program. It consists of instructions that the Python runtime can execute. Think of it as a step between the code we write and the actual execution of our program. Bytecode is not the same as machine code. Machine code consists of instructions that a processor can execute directly, whereas Python bytecode is designed for a Python virtual machine. One thing worth remembering is that Python bytecode is not universally compatible across every Python implementation and version. Its format and instructions can differ. ### Why does Python use bytecode? Bytecode provides a representation that the Python runtime can execute without having to interpret the original source text from scratch for every instruction. When CPython runs a program, the process broadly involves: 1. Reading the Python source code. 2. Compiling it into bytecode. 3. Executing that bytecode using the Python runtime. The compilation and execution happen as part of running the program, so we usually don't need to perform these steps manually. However, there is an important distinction: **compiling code into bytecode does not automatically mean that a** `.pyc` **file will be saved.** That brings us to the `__pycache__` folder. ## 3\. What Are `__pycache__` and `.pyc` Files? When I ran the program that imported a function from another file, I noticed that Python automatically created a `__pycache__` folder. Inside it was a file ending in `.pyc`. So, what exactly are these? A `.pyc` file contains cached compiled Python code, usually bytecode produced by CPython. When a module such as `chai.py` is imported, CPython can compile it and cache the resulting bytecode in a `.pyc` file. The next time the module is imported, Python may reuse that cached version if it is still valid. This can save time because Python may not need to compile the module from its source code again. ### Why is the folder called `__pycache__`? The `__pycache__` folder provides a place to store cached bytecode files instead of mixing them with our regular source files. For example, our project might look like this: ```text my_project/ ├── chai.py ├── hellochai.py └── __pycache__/ └── chai.cpython-313.pyc ``` The exact filename depends on the Python implementation and version. The example above is illustrative. Keeping these generated files in a separate folder helps keep our project organized. ### Does Python create a `.pyc` file every time we run a program? Not necessarily. In our example, `chai.py` is imported by `hellochai.py`, so CPython can cache the compiled bytecode for `chai.py`. However, when we execute a regular Python script directly, Python doesn't normally save a `.pyc` file for that main script simply because we ran it. The script is still compiled to bytecode for execution. The difference is that the compiled bytecode is not necessarily written to a cache file. Also, `.pyc` files don't necessarily get regenerated every time we run a program. Python checks whether cached bytecode is usable and recompiles the source when needed. One more clarification: `.pyc` files are not usually called *frozen binaries*. They are cached compiled Python code. Frozen modules and frozen bytecode are related but distinct concepts. ## 4\. What Exactly Is the Python Virtual Machine (PVM)? This was the part that confused me the most. I understood that Python code gets compiled into bytecode, but then I wondered: **If bytecode is generated, what does the PVM actually do?** The PVM is commonly described as the component that executes Python bytecode. In CPython, the runtime contains an interpreter that processes bytecode instructions and carries out the operations they describe. For example, when we write: ```python a = 10 b = 20 print(a + b) ``` The runtime needs to perform several operations: create and assign objects, retrieve values, perform addition, and display the result. The bytecode represents the operations to be performed, and the interpreter executes the corresponding instructions. ### Does the PVM come with Python? When we install a Python implementation such as CPython, we get the runtime and interpreter components needed to execute Python programs. We don't normally install a separate PVM ourselves. Also, PVM is a conceptual term used to explain Python's execution model. The precise implementation details depend on which Python implementation we're using. ### Does Python execute source code directly if there are no imports? This was another point I needed to clarify. No. In CPython, the absence of imports does not mean the source code bypasses bytecode. Even a simple script such as: ```python print("Hello, Python!") ``` is compiled into bytecode for execution. The important distinction is between **compiling code for execution** and **saving compiled bytecode to a** `.pyc` **file**. A script can be compiled into bytecode and executed without saving a `.pyc` file for that script. ## 5\. How Does the PVM Execute Bytecode? One way to understand the process is to imagine the interpreter working through a sequence of instructions. Broadly, it follows a cycle: **Fetch an instruction → Execute the instruction → Fetch the next instruction → Execute the next instruction → Repeat** This continues as the program runs. The interpreter processes instructions and performs the corresponding operations. Depending on the instruction, this could involve arithmetic, calling a function, accessing an object, or controlling the program's flow. The PVM isn't simply a folder containing bytecode, nor is it the same thing as the `.pyc` file. The `.pyc` file stores cached compiled code, while the interpreter is responsible for executing the instructions. ### Why is it called a runtime engine? A runtime is the environment that supports a program while it executes. In Python, the runtime does more than process bytecode instructions. It also supports important operations such as: * **Memory management:** Managing memory used by Python objects. * **Object management:** Handling Python objects and their behavior. * **Execution state:** Keeping track of the program's current execution. * **Function calls:** Managing how functions are called and how execution moves between them. These responsibilities are part of the broader Python runtime, rather than all being performed by one single component called the PVM. Thinking of the runtime as the machinery that helps a program execute made the concept easier for me to understand. ## 6\. What Is a Python Implementation? Towards the end of the video, we were introduced to different Python implementations. At first, I was confused by the word *implementation*. I wondered how it could be different from Python's rules. A simple way to understand the distinction is this: **Python's language specification describes how Python code should behave. An implementation is the actual software that makes that behavior happen.** The language defines things such as syntax, expressions, data types, and expected behavior. An implementation provides the machinery needed to execute programs according to those rules. Different implementations can follow the same language specification while using different technologies internally. Let's look at some examples. ### A. CPython CPython is the standard and most widely used implementation of Python. It is primarily written in C, and it compiles Python source code into bytecode that its interpreter executes. When we download Python from the official Python website for a typical installation, we're generally using CPython. ### B. PyPy PyPy is another Python implementation that focuses on execution speed. One of its important features is a Just-In-Time (JIT) compiler, which can compile frequently executed code into machine code while the program is running. This can make some programs run faster than they do under CPython, although the actual performance depends on the workload. ### C. Jython Jython is a Python implementation that runs on the Java Virtual Machine (JVM). It allows Python code to work with Java classes and libraries. The JVM is the runtime environment that executes Java bytecode and provides services for Java-based applications. Jython is a distinct implementation, and its Python language-version support differs from modern CPython. ### D. Stackless Python Stackless Python is based on CPython and focuses on alternative ways of managing execution, including task scheduling and lightweight tasklets. It aims to reduce dependence on the traditional C call stack used in ordinary function calls and supports alternative approaches to concurrency. In simpler terms, it explores different ways to manage tasks and execution rather than relying entirely on the usual call-stack model. ### E. IronPython IronPython is a Python implementation designed to work with the .NET ecosystem. It allows Python code to interact with .NET libraries and technologies. But what exactly is .NET? .NET is a development platform from Microsoft that provides tools, libraries, and a runtime for building different types of applications. IronPython makes it possible to use Python in that ecosystem. ### A quick comparison | Implementation | Main technology or focus | | --- | --- | | CPython | Standard implementation, primarily written in C | | PyPy | JIT compilation and execution speed | | Jython | Java Virtual Machine and Java ecosystem | | Stackless Python | Alternative execution and task management | | IronPython | .NET ecosystem integration | These implementations demonstrate that Python is not just one particular executable program. It is a programming language that can be implemented in different ways. ## 7\. My Key Takeaways Here's what I took away from this video: 1. **Bytecode** is an intermediate representation of Python code that the runtime can execute. It is not machine code. 2. `__pycache__` is a folder where Python can store cached compiled bytecode for imported modules. 3. `.pyc` **files** contain cached compiled Python code and may be reused when the cache is valid. 4. **The PVM** is commonly used to explain how Python bytecode is executed by the interpreter. 5. **The Python runtime** provides the machinery required to execute and manage a running program. 6. **A Python implementation** is the actual software that implements the language's rules. 7. **CPython, PyPy, Jython, Stackless Python, and IronPython** are examples of different implementations, each with its own approach or focus. ## Final Thoughts This video made me realize that running a Python program involves more than simply giving instructions to the computer. There's source code, bytecode, an interpreter, cached compiled files, and a runtime environment working together to make execution possible. The concept that took me the longest to understand was the PVM. Initially, I thought bytecode was something generated only when imports happened. Now I understand that compilation for execution and caching compiled code are two different things. I also learned that Python's language rules and its implementation aren't the same. The rules describe how the language should behave, while an implementation provides the software that makes it work. I'm still learning these concepts, but connecting them to an actual program made them much easier to explore. One question leads to another, and that's exactly what makes this journey interesting. * * * *This is part of my Python learning journey, where I document what I understand from each Chai aur Code lecture, the questions I encounter, and the concepts I explore along the way.* **Have you explored how Python works behind the scenes?** If you have a simpler explanation, a useful resource, or a correction to something I've misunderstood, I'd love to learn from you in the comments.