How to Install Python and Requests for Beginners

This page is the first practical step in Setting Up Your Python Scraping Environment: getting a Python interpreter, a virtual environment, and the requests library onto a machine and proving that all three actually talk to each other.

Four steps from a fresh machine to a working scraping environment.

Install Python 3.10 or newer from python.org (Windows and macOS) or your distribution's package manager (Linux), create a project-local virtual environment with python3 -m venv .venv, activate it, and install the library with python -m pip install requests. Then verify with four commands rather than one — most "it didn't install" reports are actually a working install attached to a different interpreter than the one being run. The whole sequence takes under ten minutes on a machine with a working internet connection.

Why Version and Environment Choice Matters Before You Install

Python 3.10 is the practical floor for modern scraping code. It introduced structural pattern matching and, more relevantly for library code, the X | Y union syntax in annotations at runtime, which means typed signatures like def fetch(url: str) -> str | None: work without a from __future__ import annotations line. Python 3.12 removed the bundled distutils module, which broke a great many packages that imported it during their build step; by 3.13 nearly all of them had migrated, but if you are installing a pinned old dependency set you may still meet it. As of mid-2026 the safe default is the newest 3.13.x patch release: it has wheels published for every library in this guide, and it is not so new that packages lag behind it.

The second decision is where packages go. A system-wide pip install writes into the interpreter that ships with your OS. On Debian and Ubuntu that interpreter is managed by apt, and since Python 3.11 those distributions mark it as externally managed — a bare pip install requests there fails immediately with error: externally-managed-environment. That error is not a bug; it is the packaging system refusing to let pip overwrite files apt owns. The fix is always the same: install into a virtual environment you own.

A virtual environment is a directory containing a pyvenv.cfg file, a bin/ (or Scripts/ on Windows) folder with a symlinked or copied interpreter, and a private site-packages. Nothing magic happens on activation: the activate script prepends that bin/ directory to PATH and sets VIRTUAL_ENV. That is the entire mechanism, and knowing it makes every later "wrong pip" problem diagnosable.

Installing Python on Each Operating System

Windows. Download the 64-bit installer from python.org and tick Add python.exe to PATH on the first screen. If you miss it, do not reinstall — Windows also ships the py launcher, which reads shebang lines and version flags, so py -3.13 -m venv .venv works even when python is not on PATH. Windows additionally installs App Execution Aliases for python and python3 that open the Microsoft Store; if typing python opens the Store instead of a REPL, turn those aliases off under Settings → Apps → Advanced app settings → App execution aliases.

macOS. The system /usr/bin/python3 is an Xcode command-line-tools stub, and Apple explicitly does not support using it for application development. Install a real interpreter instead:

brew install python@3.13
python3.13 --version

Homebrew places the binary in /opt/homebrew/bin on Apple silicon and /usr/local/bin on Intel. Both are ahead of /usr/bin in a default PATH, so python3 will resolve to the Homebrew build once the shell is restarted.

Linux. Use the distribution package plus the venv module, which Debian and Ubuntu split into a separate package:

sudo apt update
sudo apt install -y python3 python3-pip python3-venv
python3 --version

Omitting python3-venv produces a confusing failure later: python3 -m venv .venv exits non-zero with ensurepip is not available. Install the package and re-run the command; there is nothing to clean up first.

Creating the Environment and Installing requests

Run these four commands from the project directory. The activation line is the only one that differs by platform.

python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install requests

Note the python -m pip form rather than bare pip. Inside an activated environment both usually resolve to the same place, but the moment activation silently fails — a new terminal tab, a cron job, an IDE run configuration that ignores your shell — bare pip falls back to whatever comes first on PATH, and the package lands somewhere your script will never look.

Bare pip is resolved by PATH and can belong to another interpreter. python -m pip is resolved by the interpreter you just invoked, so the package always lands where you will import it.

requests is not a single package. Installing it pulls four runtime dependencies: urllib3 (the connection pooling and retry layer that does the actual socket work), certifi (the bundled CA certificate store), charset-normalizer (encoding detection for response.text), and idna (internationalised domain name encoding). Pin them in a lockfile for anything you will run more than once:

python -m pip freeze > requirements.txt

Two version notes matter for scraping. requests 2.32 tightened certificate handling and changed how verify= interacts with custom adapters, so proxy setups written against 2.28 sometimes need adjusting. And urllib3 2.x dropped support for OpenSSL below 1.1.1, which is why installs on very old CentOS or Amazon Linux 1 images silently resolve urllib3 back to the 1.26 line.

Verifying the Install in Four Steps

Do not verify with a single command. Each check below fails for a different reason, and knowing which one broke tells you exactly what to fix.

Four checks, in order. Each one isolates a different failure: no interpreter, the wrong pip, an inactive environment, or a missing package.

Save this as verify_setup.py and run it with the interpreter from your environment:

"""Prove the interpreter, the environment and requests all line up."""
import sys

import requests


def check() -> None:
    print(f"interpreter : {sys.executable}")
    print(f"version     : {sys.version.split()[0]}")
    print(f"env prefix  : {sys.prefix}")
    print(f"requests    : {requests.__version__}")

    response = requests.get(
        "https://httpbin.org/get",
        headers={
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
            "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36",
            "Accept": "application/json",
        },
        timeout=15,
    )
    response.raise_for_status()
    print(f"status      : {response.status_code}")
    print(f"seen by srv : {response.json()['headers']['User-Agent'][:40]}")


if __name__ == "__main__":
    check()

python verify_setup.py

A healthy run prints an env prefix ending in .venv, a requests version, a 200 status, and the User-Agent string the server actually received. That last line is the one beginners skip and later regret: it proves your headers reach the server intact, which is the foundation of everything in Understanding HTTP Requests and Responses. The default header, had you not set one, would be python-requests/2.32.x — a string many sites filter on directly.

The timeout=15 argument is not decoration. requests has no default timeout; without it a hung server leaves your script blocked on a socket read indefinitely, and a scraper that stalls at 3 a.m. with no traceback is far harder to debug than one that raises requests.exceptions.ReadTimeout.

How Python Decides Which requests It Imports

When the import requests line executes, Python walks sys.path in order and takes the first directory that contains a matching module. Inside an activated environment that list starts with the script's own directory, then the standard library, then the environment's site-packages. The system site-packages is not on the list at all unless the environment was created with --system-site-packages. This ordering explains two failures that look unrelated but share one cause.

The first is a file named requests.py (or email.py, or json.py) sitting next to your script. Because the script's directory is searched before anything else, your file shadows the real library and the traceback points at a line inside your own file — AttributeError: module 'requests' has no attribute 'get'. Rename the file and delete the stale __pycache__ directory beside it.

The second is an editor that runs code with a different interpreter than your terminal. VS Code, PyCharm and Jupyter each keep their own interpreter setting, and none of them read the PATH your shell exported. When code works in the terminal but raises ModuleNotFoundError in the editor, print sys.executable from both and point the editor at the .venv binary.

import sys

for entry in sys.path:
    print(entry or "<script directory>")

Running that inside the environment should show a .venv path before any system directory. If a system path appears first, the environment is not really active regardless of what the shell prompt says.

Pinning the Environment So It Rebuilds Identically

pip freeze captures exactly what is installed, including transitive dependencies, which is what you want for a scraper that must behave the same next month. Keep the direct requirements and the resolved lock separate:

python -m pip install requests beautifulsoup4 lxml
python -m pip freeze --exclude-editable > requirements.lock
python -m pip install -r requirements.lock

Rebuilding elsewhere is then a two-line operation, and a dependency that silently changes behaviour — charset-normalizer altering its detection heuristics, for instance, which changes what response.text produces on an ambiguous page — shows up as a diff rather than as mysterious data corruption. Delete and recreate the whole .venv directory whenever it gets into a strange state; nothing outside it is affected, and recreating takes seconds.

Edge Cases and Caveats

error: externally-managed-environment on Debian, Ubuntu, Fedora and Homebrew Python. Create a virtual environment. Reach for --break-system-packages only on a throwaway container, never on a machine you care about.
ModuleNotFoundError: No module named 'requests' right after a successful install. The install went to a different interpreter. Run python -m pip show requests and compare its Location: line with python -c "import sys; print(sys.prefix)". If they disagree, reinstall with python -m pip.
SSLCertVerificationError: unable to get local issuer certificate on macOS. The stock framework build does not link the system trust store. Run the Install Certificates.command script inside /Applications/Python 3.13/, or use the Homebrew build, which links certifi correctly. Do not "fix" this with verify=False; that disables certificate checking entirely.
Corporate TLS interception. Behind a proxy that re-signs traffic, set REQUESTS_CA_BUNDLE to a PEM file containing your organisation's root certificate rather than turning verification off.
Building lxml from source on Alpine. pip prefers manylinux wheels, and Alpine's musl libc does not match them, so the install falls back to compiling. Install libxml2-dev libxslt-dev gcc musl-dev first, or use a Debian-based image.
Windows PowerShell activation refused. .venv\Scripts\Activate.ps1 fails under the default execution policy. Run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned, or use .venv\Scripts\activate.bat from cmd.exe.
.venv committed to git. Environments are machine-specific and contain absolute paths in pyvenv.cfg; add .venv/ to .gitignore and commit requirements.txt instead.

Frequently Asked Questions

Do I need Anaconda, or is plain Python enough for scraping? Plain Python with venv is enough and is what most scraping code assumes. Anaconda is useful when your pipeline also involves compiled scientific stacks with awkward binary dependencies, but for requests, BeautifulSoup and lxml the standard wheels install cleanly everywhere, and mixing conda install with pip install in one environment is a common source of broken dependency resolution.

Why does pip install succeed but the import still fail? Because pip and python resolved to different interpreters. A bare pip is just the first executable named pip on your PATH, which is frequently the system Python rather than your project environment. Always install with python -m pip install <package> so the interpreter you are running chooses the target, and confirm with python -m pip show requests.

Can requests scrape a page whose content is rendered by JavaScript? No. requests performs an HTTP transaction and hands back the bytes the server sent; it has no JavaScript engine and no DOM. If the data you want is absent from response.text but visible in the browser, either find the underlying JSON endpoint in the network panel or drive a real browser, as covered in Using Playwright for Modern Web Automation.

Should I use requests or httpx for a first project? Start with requests. Its synchronous API is simpler, nearly every tutorial and Stack Overflow answer targets it, and the concepts transfer directly. Move to httpx when you need HTTP/2 or concurrency, which is the subject of Asynchronous Scraping with Asyncio and HTTPX.

Setting Up Your Python Scraping Environment — the parent topic, covering project layout and dependency pinning.
Understanding HTTP Requests and Responses — what to do with the response object once the install works.
Parsing HTML with BeautifulSoup — turning that HTML into structured data.
Managing Cookies and Sessions — reusing one connection and one cookie jar across requests.