Duplicate job postings are common, because the same opening often reaches Zorlya more than once. A company posts it on its own applicant tracking system, a job board copies it, and a second board copies the first. If every copy reached your feed, a good match would fill half a screen. So Zorlya removes duplicates. It does so with two narrow rules, and this post explains both, what they miss on purpose, and where they can go wrong.
Why does the same job appear more than once?
There are several honest reasons, and only some of them are true duplicates.
| What you see | What is usually behind it | One job or several? |
|---|---|---|
| The same ad on the employer’s page and on a board | The board copied it, or the employer posted it there too | One |
| The same ad from two recruitment agencies | Each agency advertises the vacancy it was given | Usually one |
| The same title for several cities | One posting per office | Often several, sometimes one advertised widely |
| The same ad a few weeks later | The employer reposted it, or reused an old ad for a new opening | Unclear until you ask |
| The same title with a different ad | Two teams hiring for similar roles | Several |
Zorlya’s rules only touch the first kind, and only when the copies match exactly. The others are left for you to judge, because they can be real separate openings.
Rule one: the same link
Each source returns postings with a link. Inside one source’s run, Zorlya compares links after trimming spaces and trailing slashes and ignoring upper and lower case. If two postings share a link by that test, the first one is kept and the rest are dropped. The link stored is the one the source sent, unchanged, so the apply button still goes where the source said.
The database adds a second guard on the link itself. A posting whose exact link is already stored does not become a second row. It updates the first one, so a posting an employer edits is refreshed and not duplicated.
Rule two: the same content
Different links can lead to the same ad. Aggregators hand out their own link for a posting they copied from somewhere else. For that case the database computes a fingerprint of every posting from four things:
- the title, trimmed and lower-cased
- the company, trimmed and lower-cased
- the location, trimmed and lower-cased
- a hash of the full description
A unique index on that fingerprint refuses a second row with the same value. When a batch hits it, Zorlya retries the batch one posting at a time and drops only the posting that collided. The run records how many postings were skipped as identical, so the loss is counted and not silent.
The posting that arrived first stays. If a board copy is stored before the employer’s own listing is read, the board’s copy is the one in your feed. Zorlya does not currently swap in the original afterward.
What it deliberately does not do
Neither rule looks for near matches. Four examples of postings that stay separate:
- A different company spelling. “Acme” and “Acme Inc” give different fingerprints, because the company is compared as written.
- A shortened description. A board that cuts the ad to a few lines produces a different description hash than the employer’s full text. The existing sources post describes why employer boards are preferred: they carry the whole ad.
- A different location. The same ad posted for Berlin and for Lisbon is two jobs, because location is part of the fingerprint. Often that is right, since they are two openings.
- One edited word. Change a sentence in the description and the hash changes.
Zorlya also matches company names to a single employer record, which is a separate step from removing postings. That match lowercases the name, turns punctuation into spaces and removes trailing legal endings such as Inc, LLC, Ltd and GmbH, so “Acme Inc” and “Acme” resolve to one company with one logo. It links the postings to the company. It does not merge them.
The reason for the narrow rules is the cost of the opposite mistake. A loose rule that merged “close enough” postings would sooner or later hide a real opening that only looked like another one. A duplicate costs you a second of attention. A hidden job costs you the job.
A worked example
Illustrative: the employer, the links and the ads are invented. Postings 1 and 2 arrive in one run of the employer’s hiring system, and postings 3 and 4 come later from a board:
| # | Link | Title, company, place | Description | Result |
|---|---|---|---|---|
| 1 | example-ats.com/acme/123 | Data Analyst, Acme, Lisbon | The full ad | Stored |
| 2 | Example-ATS.com/acme/123/ | Data Analyst, Acme, Lisbon | The full ad | Dropped: within the run, the same link once case and the trailing slash are ignored |
| 3 | board.example/job/9876 | Data Analyst, Acme, Lisbon | The full ad, word for word | Dropped: the same fingerprint |
| 4 | board.example/job/9877 | Data Analyst, Acme Inc, Lisbon | The full ad | Kept: the company is spelled differently |
Posting 4 shows in your feed next to posting 1. Both cards link to Acme as one company, because the company match removes “Inc”, but they are two jobs.
What can go wrong
Two failures are possible, and they pull in opposite directions.
A duplicate survives. This is the common one. Any copy that differs in title wording, company spelling, location text or description will pass both rules. If you see the same role twice, the two cards will usually show different sources. Compare the apply links, and prefer the one on the employer’s own domain.
Two real jobs become one. If an employer posts several openings with the same title, company, location and ad text but separate links, the content rule keeps only the first. The links differ, so rule one would keep them all, but the fingerprint is stricter and wins. Postings with an empty company are exempt from the content rule, so they are only checked by link.
How to use this
When you scan a feed, a repeated title from the same company is a signal to check the source on each card. Pick the one closest to the employer. If a role you expected is missing, a stored near-copy under another company spelling is not the cause, because near copies are kept. The more likely reasons are the date window, your filters, or that no source Zorlya reads has the posting.
Does a duplicate change my score or my tracking?
Each copy is a job of its own. It has its own score, and a copy with a shortened description may score lower than the full ad, because the score has less text to compare with your resume. Saving or tracking one copy does not touch the other. If you apply through one, mark that one in your tracker and ignore the other.
Which copy should I apply through?
- Open both links.
- Prefer the one on the employer’s own site or its hiring system.
- If both are boards, prefer the one that names the employer and shows the full ad.
- If an agency’s copy is the only one, apply there, and ask the agency which employer it is for.
- Apply once. Two applications to the same opening, through two routes, can confuse the employer about who referred you.
Questions people ask
Why does Zorlya not merge similar postings?
Because a loose rule would sooner or later hide a real opening that only looked like another one. A duplicate costs you a second. A hidden job costs you the job.
Is the same role in three cities one job or three?
It can be either. Read each ad for the number of openings and the place, and ask the employer if it matters to you.
Can a reposted ad be a new opening?
Yes. Some employers reuse an ad when they hire for the same role again. Check the date and the reference number on the employer’s own page.
Which copy does Zorlya keep?
The one stored first. If a board’s copy arrived before the employer’s own listing, the board’s copy is the one in your feed.
What we know and what we do not
We know the two rules from the code that runs them. Zorlya does not publish a duplicate rate, and there is no study behind this post. The run counts how many postings it skipped as identical, but nobody has compared those counts against a hand-labelled sample to say how many duplicates still get through, or how many were wrongly dropped. Until that sample exists, treat the two rules as a description of the behaviour and not a promise about accuracy.