2026-09-23·by Sijie Wang#node#project#youteacher#job_scrapers

platform-strategy-factory-pattern

Adding a platform: the strategy + factory pattern

In job_scrapers, each job board is a platform, and every platform is reached through two pluggable pieces: a scraper that walks listing pages and a parser that turns one job's HTML into structured data. Both are chosen at runtime by name, so adding a board means writing two classes and registering them — never editing the code that drives the scrape.

Two base classes, two jobs

ScraperStrategy is the abstract base for scrapers. It holds a ScraperConfig (platform, baseUrl, optional rateLimit and timeout) and forces one method: scrape(browser), typed as an AsyncGenerator that yields job listings one at a time. The generator shape is deliberate — it lets each platform decide internally when to fetch the next job and keeps memory flat instead of buffering a whole board. The base also carries a sleep() helper for rate limiting and a getPlatform() accessor.

ParserStrategy is the abstract base for parsers. Its one required method, parse(html, jobId, sourceUrl, existingJobDescription?, rawData?), returns a Promise<ParsedJob>. The contract described on the class is a three-step flow: extract what the DOM gives up, call AI to fill the missing fields, then merge. The optional existingJobDescription lets a parser skip re-parsing when the job description hasn't changed, and rawData is there for platforms that hand back JSON rather than HTML. Shared helpers live on the base: validateRequiredFields (which insists on a non-empty jobTitle and jobDescription), cleanText, normalizeText (used for description comparison), and an enrichFromDescription hook that defaults to doing nothing.

Two factories, two styles

ScraperFactory is registry-driven. A single STRATEGIES array lists each platform as { id, name, factory }, and the factory builds a Map keyed by id from it. Lookups lowercase the incoming name, so casing is forgiving. On top of that map it offers createScraper (throws if the platform is unknown), normalizePlatformName (arbitrary casing/alias → canonical name, or null), getSupportedPlatforms, and isSupported. Adding a scraper is one line in the array.

ParserFactory is switch-driven instead: createParser matches the lowercased platform in a switch and news up the matching parser, throwing on the default branch. It carries a couple of parser cases (echinacities, schrole) that the scraper registry doesn't list — the two sides don't have to enumerate an identical set.

Which platforms are wired

Six scrapers are registered in STRATEGIES: DavesESL, TEAST, EChinaCareers, TeachAway, SearchAssociates, SeekTeachers.

Registering a new platform

  1. Write a scraper class extending ScraperStrategy and implementing scrape().
  2. Write a parser class extending ParserStrategy and implementing parse().
  3. Add a { id, name, factory } entry to STRATEGIES in ScraperFactory.ts.
  4. Add a case to the switch in ParserFactory.ts.

No caller changes: everything downstream already resolves scrapers and parsers by name.

about this entry

One of sijie's wiki entries. The AI on this site is grounded in the same corpus and answers in sijie's voice, with citations back to entries like this one — answering costs sijie money, so it waits behind a code: enter an access code →