LLM Job-Skill Extraction

The question

“An LLM-Based Approach for Extracting In-Demand Job Skills to Enable AI-Powered Curriculum Enhancement” — my Master's thesis at ITC, with the core of the work done during a three-month research stay at the ICTEAM institute, UCLouvain, funded by the ARES mobility programme.

Two questions drive it. How accurately can an LLM pull structured skill information out of noisy HTML job pages? And how can curriculum data be mapped into that same skill space, so the two are actually comparable rather than merely adjacent?

Labour-market intelligence

The first framework discovers country-specific job portals, crawls postings at scale, and extracts the skills each one requires using a qwen3 model run locally through Ollama under a constrained output schema — the schema is what makes the output parseable rather than merely plausible.

The resulting inventory is then analysed for what is in demand, what is trending, and what is newly emerging, so the picture reflects how requirements move over time rather than a single snapshot.

Curriculum as competencies, not keywords

The second framework reads each course's learning outcomes and lesson plan, extracts the skills it actually teaches, and groups them into discipline-native competencies — deliberately without reference to any market vocabulary. The curriculum gets described in its own terms before anything is compared, so the comparison is not quietly rigged by borrowing the market's words first.

Retrieve, then verify

Embedding similarity alone produces confident false matches: two skills can sit close in vector space and mean different things. So matching runs in two stages, each tuned for what it is good at — embedding-based retrieval shortlists candidates for high recall, then an LLM verification pass confirms genuine coverage for high precision.

That split is what makes the alignment figures trustworthy. A single-stage matcher would have reported a curriculum as well-covered on the strength of near-misses.

Evidence, not just a score

Each course receives a demand-weighted alignment score and a verdict — maintain, revise, or update — and every verdict carries the evidence and reasoning behind it.

That last part is the difference between a result and a usable tool. A bare number does not survive a curriculum committee; the specific gap between what is taught and what is demanded, with its supporting evidence attached, does.

Evaluation

Extraction quality was validated against a human-annotated benchmark, scoring 78.0% semantic F1 and 97.2% Precision@5 on SkillSpan — the extraction rarely surfaces a skill that is not there, which matters more than raw recall when the output is meant to inform curriculum decisions.

Controlled ablations justify the design choices rather than asserting them, and the diagnostic work throughout was about separating data problems from method problems from model problems — three failures that look identical in an end-to-end score.

Where it stands

Presented at ITC's scientific day in June 2026, with a paper prepared for the Techno-Science Research Journal. Thesis defence is September 2026.