XPath vs CSS Selectors: Which Should You Use?

This page settles the choice between the two selector languages introduced in Selecting Elements with XPath and CSS Selectors, with the capability matrix, the compilation path, and a measurement instead of an opinion.

Start with CSS. It is shorter, it reads like the stylesheets the site's own developers wrote, and it covers class, id, attribute, child and descendant matching — which is most of every extraction job. Switch to XPath for exactly two things CSS cannot do: matching on the visible text of an element, and moving upward or backward through the tree. Performance is not a tie-breaker, because in lxml a CSS selector is translated into XPath before it runs.

Why the Performance Question Is Already Answered

The claim "XPath is faster than CSS" is a category error in Python. lxml has one query engine: libxml2's XPath implementation. When you call .cssselect(), the cssselect package parses your CSS selector, rewrites it as an XPath expression, and hands that expression to the same engine. There is no second code path to be faster than.

You can see the rewrite directly:

from cssselect import HTMLTranslator

translator = HTMLTranslator()
for selector in ["ul > li", "div p", "#main", "a.title"]:
    print(f"{selector:12} -> {translator.css_to_xpath(selector)}")

That prints, among other things, ul > li -> descendant-or-self::ul/li and the considerably less pretty expansion of a.title into a contains(concat(' ', normalize-space(@class), ' '), ' title ') predicate. The expansion is not cssselect being clumsy — it is the correct way to test one token inside a space-separated attribute, and a hand-written XPath class match has to do the same work to avoid matching class="titlebar".

The cost of that translation is paid once per unique selector string, and it is a few tens of microseconds. The cost of the query itself is identical for both languages because it is the same query. Against that, a single HTTP request to a real site costs somewhere between 80 and 800 milliseconds. Selector language is not a term in the throughput equation.

BeautifulSoup is the one place where the picture changes, and not in XPath's favour: BeautifulSoup has no XPath engine at all. Its .select() method uses soupsieve, a separate and very complete CSS implementation that walks BeautifulSoup's own object tree. If you are working inside Parsing HTML with BeautifulSoup, CSS is not merely preferable, it is the only selector language available without reparsing the document with lxml.

Measuring It Yourself

Rather than trusting the argument, measure it. The script below fetches one real page, parses it once, and times an equivalent CSS selector and XPath expression over the already-parsed tree, so the only thing being compared is query execution.

import timeit

import requests
from lxml import etree, html

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-GB,en;q=0.9",
}

URL = "https://books.toscrape.com/catalogue/category/books/fiction_10/index.html"
response = requests.get(URL, headers=HEADERS, timeout=20)
response.raise_for_status()
tree = html.fromstring(response.text)

compiled = etree.XPath(
    "//article[contains(concat(' ', normalize-space(@class), ' '), ' product_pod ')]"
)

runs = 2000
css = timeit.timeit(lambda: tree.cssselect("article.product_pod"), number=runs)
xpath = timeit.timeit(lambda: tree.xpath("//article[@class='product_pod']"), number=runs)
precompiled = timeit.timeit(lambda: compiled(tree), number=runs)

print(f"network fetch      {response.elapsed.total_seconds() * 1000:8.2f} ms (once)")
print(f"cssselect          {css / runs * 1e6:8.2f} us per call")
print(f"tree.xpath         {xpath / runs * 1e6:8.2f} us per call")
print(f"compiled XPath     {precompiled / runs * 1e6:8.2f} us per call")

On a mid-range laptop the three query numbers land in the tens of microseconds and within a small multiple of each other, while response.elapsed reports a figure three to four orders of magnitude larger. The precompiled expression is the fastest of the three because it skips expression parsing, which is the only optimisation in this area that ever shows up in a profile — and it applies equally to CSS, since cssselect will happily give you an XPath string to compile.

Two caveats about that benchmark. It reuses one parsed tree, so it excludes parsing, which genuinely does cost milliseconds and genuinely does differ between parsers — that comparison lives in BeautifulSoup vs lxml: Which Parser Is Faster. And it uses a small document; on a 2 MB page, an unanchored // scan costs proportionally more than a scoped .// search in either language, which is a structural fact about the query, not about the syntax you wrote it in.

Readability and Team Maintainability

With speed off the table, the decision is about what the expression communicates to the next person who reads it.

The first three rows are why CSS is the default: the CSS form is the one a front-end developer would recognise instantly, and shorter expressions produce shorter diffs when a site changes. The last two rows are why every non-trivial scraper contains some XPath: no amount of CSS cleverness reaches a parent.

There is a real team cost to mixing them arbitrarily. A codebase where half the selectors are CSS and half are XPath, chosen by whoever wrote them that day, is harder to audit than one with a rule. A workable rule is the one drawn below: CSS unless the field needs a text predicate or an axis, and a short comment on every XPath explaining which of those two it is.

Mixing is cheap in Web Scraping with Scrapy, because response.css() and response.xpath() return the same SelectorList type and can be chained in either order. Scrapy also adds ::text and ::attr(name) to CSS, which removes the most common reason people abandon CSS mid-expression.

from scrapy.selector import Selector

sel = Selector(
    text="""
    <article class="product_pod" data-sku="a897fe">
      <h3><a title="Nod on the Tune">Nod on the Tune</a></h3>
      <p class="price_color">£24.00</p>
      <span class="badge">Sale</span>
    </article>
    """
)

# CSS for the container, XPath for the relative step.
print(sel.css("article.product_pod").xpath("./h3/a/@title").getall())

# XPath for the anchor-then-climb, CSS for the field inside the result.
print(sel.xpath('//span[@class="badge"]/ancestor::article[1]').css("::attr(data-sku)").get())

# Scrapy's CSS extensions cover the two things plain CSS cannot return.
print(sel.css("p.price_color::text").get())

Chaining in either direction works because each call re-roots on the previous result. The one asymmetry to remember: chained CSS is relative automatically, chained XPath is not. card.xpath("//p") searches the whole document even when card is a single card, so the leading . in .//p is mandatory.

Where XPath Earns Its Keep

The two capabilities CSS lacks are not exotic. Both appear on ordinary commerce and reference pages.

Anchor-then-climb. You can identify a badge, an icon or a label reliably, but the value you want lives on an ancestor. ancestor::article[1] walks up to the nearest matching container; the [1] means "the closest one", because the ancestor axis is ordered from the context node outwards.

Label-then-step-sideways. Specification tables have no per-field hooks — every cell is a td. //th[normalize-space()="UPC"]/following-sibling::td[1] finds the value by the only stable thing on the row, its label. preceding-sibling:: handles the mirror-image layout where the label follows the value.

from lxml import html

tree = html.fromstring(
    """
    <article data-sku="a897fe">
      <span class="badge">Sale</span>
      <table>
        <tr><th>UPC</th><td>a897fe39b1053632</td></tr>
        <tr><th>Availability</th><td>In stock (22 available)</td></tr>
      </table>
    </article>
    """
)

print(tree.xpath('string(//span[@class="badge"]/ancestor::article[1]/@data-sku)'))
print(tree.xpath('normalize-space(//th[normalize-space()="UPC"]/following-sibling::td[1])'))
print(tree.xpath("(//tr)[last()]/td/text()"))

The third line shows a third, smaller capability: document-wide indexing. (//tr)[last()] is the final row on the page; //tr[last()] is the final row of each table, which is a different node set entirely. CSS has :last-child and :last-of-type, both of which are per-parent, so "the last row on the page" has no CSS spelling at all.

CSS Support Is Not the Same in Every Library

"Use CSS" is one instruction with three implementations behind it, and their feature sets differ.

:has() is the entry that changed the calculus. Since cssselect 1.2 you can write article:has(span.badge) and it translates to descendant-or-self::article[descendant::span[...]], so selecting a parent by what it contains no longer requires XPath. It is still not a general upward move — :has() tests a container against its own subtree, whereas ancestor:: starts from the interesting node and walks out, which is what you want when the interesting node is what you found first.

The differences fail in two ways. p:-soup-contains("x") raises ExpressionError in cssselect, and li:not(.a, .b) raises SelectorSyntaxError — both loud, both easy to diagnose. Pin the versions of cssselect and soupsieve in your requirements file if selectors are shared between two codebases, because a selector that parses on your machine and not in production is a much worse failure than either of these.

Edge Cases and Caveats

• Namespaced documents. XHTML, SVG and RSS put elements in a namespace. //item matches nothing in an RSS feed; you need a namespaces= mapping or //*[local-name()="item"]. cssselect has namespace support but the ergonomics are worse, so XML feeds are one place XPath wins on convenience.
• :-soup-contains() is not standard CSS. soupsieve offers a text-matching pseudo-class, but it exists only in that library. Copying such a selector into lxml or a browser console fails. If a selector needs to be portable between your scraper and DevTools, XPath is the safer text-matching choice.
• :has() is still not an ancestor axis. It selects the container, so the container has to be the thing you can name. When the only thing you can name is a deeply nested badge and you want whatever article encloses it, ancestor::article[1] is the expression that exists.
• DevTools "Copy selector" output is a trap in both languages. Chrome generates positional chains full of nth-child and an invented tbody. The tbody does not exist in an lxml tree, so the copied selector fails immediately; the nth-child positions break on the next layout change.
• XPath 1.0 only. lxml implements XPath 1.0, so matches(), ends-with() and lower-case() from XPath 2.0 are unavailable. Case-insensitive matching has to be built from translate(), as covered in Selecting Elements by Visible Text in Python.
• Attribute values, not just elements. XPath can return strings directly (//a/@href, string(//h1)); CSS always returns elements, and you read the attribute in Python afterwards. For bulk attribute harvesting XPath produces less code.

Frequently Asked Questions

Is XPath actually faster than CSS in Python? No. In lxml, cssselect translates your CSS selector into an XPath expression and the same libxml2 engine executes it, so the query cost is identical and the translation adds tens of microseconds once per selector string. Any difference is invisible next to the network request, which is typically a hundred thousand times larger.

Can I use XPath with BeautifulSoup? Not directly, because BeautifulSoup ships no XPath engine. It supports CSS through soupsieve via .select() and .select_one(). If a field needs an axis or a text predicate, parse that document with lxml.html.fromstring() as well and use XPath there.

Which should I teach a new team member first? CSS, because it transfers from front-end work and covers most fields with less syntax. Introduce XPath as a targeted tool for two named situations — matching visible text and climbing to a parent — rather than as an alternative dialect for everything.

Does the choice change for JavaScript-rendered pages? Only in which API you call. Playwright and Selenium both accept CSS and XPath, and Playwright additionally offers text and role locators that are usually more readable than either. The underlying trade-off is unchanged: CSS for structure, XPath when you must match text or move upward.

Related

• Selecting Elements with XPath and CSS Selectors — the parent topic, with the step-by-step method
• Writing Selectors That Survive a Redesign — choosing anchors that outlive a front-end rebuild
• Using Playwright for Modern Web Automation — when the element only exists after JavaScript runs