How to rotate user agents per request in Node.js
If you are cycling through User-Agent strings to spread your requests out and the target still flags them, the problem is usually not the rotation. It is that the rotated User-Agent arrives next to headers that contradict it: a Chrome UA paired with Firefox's Accept order, or Sec-Ch-Ua client hints that name a different browser than the UA string does. A detector that compares the User-Agent against the rest of the header set reads that mismatch as automation, and rotating the UA alone makes it worse, because each request now carries a fresh inconsistency.
The fix is to rotate a pool of full browser identities rather than bare strings, and to send the header set that the chosen browser would actually send. We'll build a small script that picks a browser identity per request so common desktop browsers come up more often than rare ones, reads the browser family off the chosen User-Agent, and emits a User-Agent plus an Accept, Accept-Language, Accept-Encoding, and (for Chromium) Sec-Ch-Ua set that all agree with it, so the rotation never splits a UA from its matching headers. It comes to about 75 lines of Node.js with intoli/user-agents for the weighted UA pool and a small lookup that pairs each browser family with its matching headers. Rotating identities does not grant authorization or bypass a site's policy; it only keeps your authorized requests from being flagged on a header inconsistency.
The complete script
// rotate-user-agents.mjs
import UserAgent from 'user-agents'
/* one header profile per browser family. the User-Agent picks the family,
and these headers are the ones that family actually sends. keeping them
together is the point: a Chrome UA must not arrive with Firefox headers. */
const HEADER_PROFILES = {
chrome: {
accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
acceptLanguage: 'en-US,en;q=0.9',
acceptEncoding: 'gzip, deflate, br, zstd',
// client hints restate the brand + major version. Chromium sends these.
secChUa: '"Chromium";v="126", "Google Chrome";v="126", "Not.A/Brand";v="24"',
secChUaMobile: '?0',
secChUaPlatform: '"Windows"'
},
firefox: {
accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
acceptLanguage: 'en-US,en;q=0.5',
acceptEncoding: 'gzip, deflate, br',
// Firefox does not send Sec-Ch-Ua. adding it here is itself a tell.
secChUa: null
}
}
// resolve the family by its discriminating token, checked in order. Chrome's
// UA also contains "Safari" and "AppleWebKit", so a single naive test is wrong;
// check Firefox/ then Chrome/, and return null for anything else.
const familyOf = (uaString) => {
if (uaString.includes('Firefox/')) return 'firefox'
if (uaString.includes('Chrome/')) return 'chrome'
return null
}
// intoli/user-agents samples by real-world frequency. constrain to desktop
// Chrome and Firefox so every pick has a profile in HEADER_PROFILES.
const source = new UserAgent([
/Chrome|Firefox/,
{ deviceCategory: 'desktop' }
])
// build the matching header set for one chosen identity.
const headersFor = (uaString) => {
const family = familyOf(uaString)
if (!family) return null // unknown family: skip rather than send a guess
const profile = HEADER_PROFILES[family]
const headers = {
'User-Agent': uaString,
'Accept': profile.accept,
'Accept-Language': profile.acceptLanguage,
'Accept-Encoding': profile.acceptEncoding
}
if (profile.secChUa) {
headers['Sec-Ch-Ua'] = profile.secChUa
headers['Sec-Ch-Ua-Mobile'] = profile.secChUaMobile
headers['Sec-Ch-Ua-Platform'] = profile.secChUaPlatform
}
return headers
}
// one self-consistent identity per request, fresh on each call.
const urls = [
'https://httpbin.org/headers',
'https://httpbin.org/headers',
'https://httpbin.org/headers'
]
for (const url of urls) {
const headers = headersFor(source().toString())
if (!headers) continue // no profile for this family: skip, do not guess
const res = await fetch(url, { headers })
const body = await res.json()
console.log(body.headers['User-Agent'])
console.log(body.headers['Sec-Ch-Ua'] ?? '(no client hints)')
console.log('---')
}npm install user-agents
node rotate-user-agents.mjsHow it works
Keep one header profile per browser family. HEADER_PROFILES holds the Accept, Accept-Language, Accept-Encoding, and client-hint values that Chrome and Firefox each send. These are stored together so the rotation does not split a User-Agent from its matching headers. The Chrome profile carries Sec-Ch-Ua; the Firefox profile sets it to null, because Firefox does not send client hints and adding them would itself be the inconsistency a detector flags. Two static profiles cover the desktop case here; when you need header sets generated across many browser and version combinations, use Apify's header-generator, which samples matching headers from a model trained on real traffic. One more thing drifts over time: the v="126" in secChUa names a major version, so if you bump the Chrome version in a User-Agent string without updating it, the client hint and the UA disagree, and the package itself can ship stale versions if you leave it months out of date, so update it periodically and keep these major versions in step with it.
Resolve the family by an ordered token check, then key into the profile table. familyOf checks for Firefox/ before Chrome/, because a Chrome User-Agent also contains the tokens Safari and AppleWebKit, so a single naive substring test reads the family wrong and pairs a real Chrome UA with Firefox headers, the exact failure this page exists to prevent. The resolved family then indexes HEADER_PROFILES. An unrecognized family returns null and the request is skipped rather than sent with a guessed header set.
Sample the pool by real frequency. new UserAgent([/Chrome|Firefox/, { deviceCategory: 'desktop' }]) constrains intoli/user-agents to desktop Chrome and Firefox, the two families with a profile. The library weights its picks by real-world traffic frequency, so common browsers appear more often than rare ones. Calling source() returns a fresh identity object and .toString() gives its User-Agent string.
Assemble and send one identity per request. headersFor builds the full header set for the chosen User-Agent and attaches the client hints only when the profile defines them, which keeps a Firefox identity from arriving with the Sec-Ch-Ua headers it never sends. The loop calls it once per URL, so each request carries a User-Agent and a header set that agree with each other, and the next request rotates to a new pick. One thing this does not change is the TLS fingerprint: Node's fetch sends the same ClientHello under every identity, so a site that fingerprints JA3 or HTTP/2 settings sees one network signature regardless of the headers, and for those targets you send the requests through a client that varies the handshake, such as the curl-impersonate family.
Use this when
You run authorized, permitted data collection (your own sites, a licensed API, or a target whose terms and robots rules allow it) and you are spreading requests across browser identities to avoid being flagged on a header inconsistency rather than to defeat a hard block.
Respect the site's robots.txt, terms of service, and rate limits before reaching for any of this.
Skip this when
Skip it when a single fixed User-Agent already returns the page (rotation adds nothing); when the block is the browser fingerprint itself rather than the headers (patch the browser, see How to patch headless Chrome to avoid detection); when the site fingerprints TLS or HTTP/2 (vary the transport with an impersonating client); and when the limit is per-IP rather than per-identity (rotate the network path through a proxy instead).