gwas summary statistics harmonization · gwas summary statistics
GWAS Summary Statistics Harmonization for Pharma Teams
November 20, 2026
31 min read
File-state acceptance reference for GWAS summary statistics harmonization in 2026: original, formatted and harmonised files, allele flips, hm_code ledger, GCST90270927 audit, and draft CNV and gene formats.

- 01Record each GWAS Catalog file as original, legacy formatted or harmonised. File state determines what the pipeline can assume about genome build, allele orientation and retained rows.
- 02Check effect direction after allele swaps. A swap reverses the sign of beta, takes the reciprocal of the odds ratio and changes the effect allele frequency to one minus its original value.
- 03Track mapping drops and harmonisation failures separately. Record whether each rate uses all input sites or only the sites carried forward after mapping.
- 04GWAS-SSF allows a z-score fallback for meta-analyses without other effect size and uncertainty measures. CNV and gene-based results have separate draft standards and dedicated ingest routes.
- 05Run file acceptance checks on every refresh and record the results. Keep sample overlap, ancestry compatibility and linkage disequilibrium reference choice under scientific review.
Executive Summary
This report is a file-state and statistical-semantics acceptance reference for teams that download public GWAS (genome-wide association study) summary statistics and want to reuse them in colocalisation, Mendelian randomisation (MR) or target prioritisation. It does not teach the methods and it does not claim that a clean file proves causality. The decision it supports is narrower: whether a given file, in a given state, with a given sidecar, can enter an analysis, and which checks an ingest pipeline must preserve.
The GWAS Catalog separates curated, significant loci from full summary statistics, and as of 17 November 2026 it listed 239,508 full summary statistics alongside 1,206,717 top associations ([1]). Full summary statistics were downloadable for 66% of journal-published GWAS in the Catalog at the time of its 2024 paper ([2]). Files exist in three states: the original submitted file, an optional legacy "formatted" file whose headers were standardised but whose data were not manipulated, and a harmonised file reported against one genome build with alleles orientated to the forward strand ([3]).
Harmonisation is not neutral. When the effect and other alleles are swapped, the beta, odds ratio, confidence interval and effect allele frequency are adjusted ([4]). In the one real study audited here (GCST90270927, harmonised in November 2023), 4,854,469 forward-strand variants (35.14% of the 13,815,944 mapped sites) carried the flipped code ([5]), and 661,936 mapped sites (4.79%) could not be harmonised, 661,899 of them because no matching reference variant existed ([6]). That is one file under one pipeline version, not a benchmark. The Catalog's paper gives an average of about 3.5% unharmonisable variants removed ([7]), and the pipeline documentation's 95.29% example success rate is illustrative only ([8]).
Three cautions shape the acceptance rules. First, the standard is moving: GWAS-SSF v1.1.0 added a z-score fallback on 26 June 2026 ([9]). Second, copy number variant (CNV) and gene-based data are not SNP data. The methods page says they should follow the beyond-SNPs standards since April 2026 ([10]), while the linked landing page calls those standards drafts that are not yet fully integrated into the submission system ([11]). Third, metadata checks do not resolve sample overlap, ancestry mismatch or linkage disequilibrium (LD) reference mismatch, which need scientific review ([12]) ([13]). Genetic support is informative for target selection, with one analysis estimating a 2.6-fold higher probability of success for supported mechanisms ([14]), but that estimate concerns portfolios, not any single file.
Full summary statistics listed in the GWAS Catalog as of 17 November 2026
Journal-published GWAS in the Catalog with downloadable full summary statistics at the time of its 2024 paper
Carried-forward sites flipped across all orientation classes in the audited study
Harmonised sites as a share of all input sites in the audited study
Introduction and Background
Pharma statistical-genetics leads and target-discovery teams increasingly pull public GWAS summary statistics into internal evidence stores. The temptation is to treat a downloaded file as a finished input. In practice a GWAS Catalog file passes through several states, each with different guarantees about genome build, allele orientation and the rows that remain. A pipeline that cannot tell these states apart can silently reverse the direction of an effect, or compute a denominator on a file from which variants have been dropped.
The Catalog draws a clear line between its curated associations and the full data. Curated associations are the significant (p<1e-5) and independent loci reported in a paper ([15]), whereas summary statistics are the full, aggregate p-values and association data for every variant analysed in a study. Only the second is useful for colocalisation, which needs the whole region, and the coloc documentation asks users to include all SNPs in a region rather than trimming by significance or minor allele frequency ([16]). Files come from several sources, including author submission, so provenance varies.
Why does this matter commercially? Human genetic evidence is one of the better-studied predictors of drug development outcomes. An early analysis reported that the share of mechanisms with direct genetic support rose from 2.0% at the preclinical stage to 8.2% among approved-drug mechanisms ([17]). A 2024 update estimated that the probability of success is 2.6 times greater for genetically supported mechanisms ([14]), and reported that this ratio is sensitive to confidence in the variant-to-gene mapping. Those are population-level associations across drug programmes. They are not evidence that any particular summary-statistics file is correct, and they depend on the upstream data quality examined here.
IntuitionLabs, a life-sciences and AI consultancy, prepared this report as an adjacent advisor on data architecture and scientific AI assurance. It is not a vendor of GWAS data, harmonisation software or any product discussed. Its perspective, stated in prose only, is that file-state handling belongs in the data platform as an explicit, testable control, with the statistical-genetics team owning scientific sign-off. No client experience is described or implied. Volatile items (software versions, API status, counts) are dated to sources opened in October and November 2026.
The scope is deliberately bounded: file states, build and effect-allele invariants, the harmonisation-code ledger, metadata and licence checks, and separate routes for SNP, CNV and gene-based data. It excludes method tutorials, biobank access and downstream scoring. No dataset-wide failure fraction was measured for this report, and none of the sources read offers a representative benchmark.
Key Changes in the Standards, Pipeline and Access Routes
From pre-GWAS-SSF to GWAS-SSF, and a revised specification in 2026
Before 2023 the Catalog accepted a minimum format called pre-GWAS-SSF. A survey of 327 public summary statistics files found 127 unique formats, one for every 2.5 files ([18]). The GWAS-SSF (GWAS summary statistics format) preprint reported that of 27,845 valid files obtained by the Catalog, about 17% lacked at least one of the new mandatory fields ([19]), and that effect allele frequency was the most commonly omitted ([20]).
GWAS-SSF was finalised and implemented in the Catalog in April 2023 ([21]). It has two parts, a tab-separated data file and a YAML metadata file ([22]). The mandatory fields are chromosome, base pair location, effect allele, other allele, one effect size, standard error, effect allele frequency (EAF) and a p-value, and they must appear in a defined order. Submissions per month rose 33% after the standard was published ([23]).
The standard has continued to change. Release v1.1.0, dated 26 June 2026, added z-score fallback support for meta-analysis studies where other effect size and uncertainty measurements are unavailable ([9]). The specification explains that some meta-analysis outputs, such as METAL output, may only provide a z-score ([24]). For an ingest pipeline the consequence is concrete: a file can now be valid GWAS-SSF with no usable standard error, so code that assumes beta and standard error together will fail or, worse, treat missing values as zero. The metadata file_type string in the Catalog's example remains "GWAS-SSF v1.0", so the specification revision a file satisfied cannot be inferred from that field alone.
Original, formatted and harmonised: three file states
Table 1 below summarises the states a pipeline will meet in the Catalog's FTP (file transfer protocol) tree. The state determines which invariants can be assumed, so the pipeline should record it as data, not infer it from a filename.
| File state | Where it sits | Genome build | Allele orientation | Sidecars and provenance | Accepted use (proposed) |
|---|---|---|---|---|---|
| Original | Accession directory, original .tsv | As declared in metadata; GCST90270927 is GRCh37 | As submitted. GWAS-SSF recommends effect and other allele fixed to ALT and REF where possible but accepts other orders | Metadata YAML with is_harmonised false; md5 checksum required at submission | Archive and audit; analyse only after your own build and allele checks |
| Formatted (legacy) | A *.f.tsv file inside harmonised/, for files harmonised before 2023 | Unchanged | Unchanged from submission | Older layout; very old files with no GCST prefix have no YAML | Header-mapping reference only |
| Harmonised | harmonised/ with .h.tsv, .tbi, .log and .yaml | One build; GCST90270927 sidecar states GRCh38 ([25]) | Forward strand, aligned to the reference | is_harmonised true; reference recorded; running log with counts | Candidate input to colocalisation and MR after the acceptance tests below |
| Awaiting or non-standard | No harmonised directory | As original | As original | Likely awaiting harmonisation, or flagged "Non-GWAS-SSF" | Quarantine; do not substitute the original silently |
The table needs one interpretive caution. The harmonised file is a different file with different rows, not the original with a column added. The pipeline documentation says that since April 2023 only the harmonised results are retained in the final .h.tsv file ([26]), while older harmonised files list harmonised values in separate hm_ prefixed columns ([27]). A reader written for one layout will misread the other, so treat the layouts as different parsers and record which one produced each row.
Genome build and coordinate invariants
The Catalog states that its data are mapped to GRCh38.p14 and dbSNP build 156, that genome build has been mandatory in summary statistics metadata for submissions since 2020, and that for older submissions the build may have been inferred during curation. The GWAS-SSF specification places the genome assembly and coordinate system in metadata, with positions following the declared system, either 1-based or 0-based ([28]), while the preprint describes the coordinate-system field as optional ([29]), so legacy files may not declare it. Chromosome values are integers from 1 to 25 ([30]).
The harmonisation pipeline maps build in an ordered sequence: it first tries to update the position by mapping the rsID against Ensembl, then falls back to liftover to the latest build, and otherwise removes the variant. The pipeline documentation names the UCSC LiftOver tool for the fallback ([31]). Two consequences follow. A position in a harmonised file may come from an rsID lookup rather than from the submitted coordinate, so rsID-to-position disagreement should be tested, not assumed away. And variants dropped as unmapped are a separate class from hm_code failures: in the pipeline's example log, 132,485 of 21,727,440 sites were dropped because they could not be mapped ([32]).
MungeSumstats can infer the genome build from the data, which offers an independent check on the declared build ([33]). The standalone harmoniser repository documents support for the Ensembl variants VCF reference (hg38, dbSNP 151) ([34]), while sidecars may name Ensembl release 95, so record both. Other resources must have their build read from each source. FinnGen documents summary statistics on genome build 38 ([35]), whereas Pan-UK Biobank flat files report positions in GRCh37 coordinates ([36]). Merging them without liftover would misalign every position. Liftover itself loses records: UCSC liftOver writes failed conversions to a separate "unMapped" output ([37]), CrossMap writes an .unmap file ([38]), and a ClinVar benchmark associated conversion failure with reference-build gaps and with variant types such as indels and duplications. A pipeline that discards the unmapped file loses the evidence of how many variants were dropped.
Effect-allele orientation and the flip arithmetic
An effect size is only meaningful relative to a named allele. MR guidance states that associations must be expressed per additional copy of the same allele when summarised data from different sources are combined ([39]), and the coloc documentation tells users to check which allele the beta describes and to switch its sign if needed ([40]). Conventions differ by source. In GWAS-VCF the alternative allele is always the effect allele ([41]), MungeSumstats assumes the second allele, A2, is the effect allele, and FinnGen ([42]) and Pan-UK Biobank ([43]) both use the alternate allele. GWAS-SSF itself does not require alignment, and its authors wrote that a mandatory requirement for allele alignment should be considered in future iterations ([44]).
Table 2 gives a clearly synthetic worked example, using separate illustrative rows so that beta and odds ratio are never mixed in one row. The values are invented and are not from any study. The arithmetic follows the documented inversion of effect size, confidence interval and EAF on a swap ([4]), and a statistician should confirm it against the specific model before it is encoded as a test.
| Illustrative row (synthetic) | Before swap (effect allele A, other allele G) | After swap (effect allele G, other allele A) | Rule applied |
|---|---|---|---|
| Row 1, continuous trait | beta 0.2, standard error 0.05, EAF 0.2, p-value 6.3e-5 | beta -0.2, standard error 0.05, EAF 0.8, p-value 6.3e-5 | beta changes sign; standard error and p-value unchanged; EAF becomes 1 minus EAF |
| Row 2, binary trait | OR 2.0, 95% CI 1.50 to 2.67, EAF 0.2 | OR 0.5, 95% CI 0.37 to 0.67, EAF 0.8 | OR becomes its reciprocal; CI bounds become reciprocals and exchange roles; EAF becomes 1 minus EAF |
Rows 1 and 2 are independent examples. In Row 2 the new lower bound is 1 divided by 2.67 (0.37) and the new upper bound is 1 divided by 1.50 (0.67). The two-sided p-value is unchanged because the test statistic changes sign but not magnitude. The invariants are testable on any harmonised file: the sign of beta (or log odds ratio) must move opposite to the swap, EAF before and after must sum to 1 within rounding, and standard error and p-value must be unchanged. Harmonisation does not re-estimate the association statistics, but when the effect and other alleles are swapped it inverts the beta, odds ratio, 95% CI and effect allele frequency, so it does not leave the signs untouched ([45]).
Harmonisation codes and the dropped-record ledger
The pipeline's reference page defines hm_code values 1 to 18 and labels codes 9 and 14 to 18 "Not harmonised", for example code 9, "Palindromic; Drop palindromic" ([46]), code 15 for no matching variant in the reference ([47]), code 16 for multiple matching variants ([48]) and code 18 for EAF below a specified minor allele frequency threshold ([49]). The numeric threshold behind code 18 was not stated in the pages read, so it must come from the pipeline version that produced the file. The methods page adds that processes involving "Infer strand" (codes 1 to 4) are currently not being used ([50]).
Palindromic variants (A/T or G/C) cannot be oriented from alleles alone. The pipeline samples 10% of non-palindromic sites to estimate strand consensus ([51]), uses a default threshold of 0.99 ([52]), and drops palindromic variants where the evidence is weak ([53]). Elsewhere the problem is handled differently: TwoSampleMR's default infers strand from allele frequencies for palindromes, and its more conservative action drops all palindromic SNPs ([54]), while TwoSampleMR also drops variants coded as D/I in one dataset and as equal-length indel alleles in the other ([55]), and MR-Base notes that allele frequency may not indicate strand when it is close to 0.5.
Table 3 proposes a retain or quarantine decision per class. The "documented meaning" column follows the pipeline's reference pages and the methods page. The decision column is this report's proposal and must be verified against the pipeline version recorded in the running log: the example log reports pipeline version 0.1.0 ([56]), and later standalone releases added features such as z-score harmonisation ([57]).
| hm_code | Documented meaning | Proposed decision | Condition to verify |
|---|---|---|---|
| 10, 12 | Non-palindromic; correct orientation (12 is reverse strand, so alleles are reverse complemented) | Retain | Count matches the log; spot-check sign against source |
| 11, 13 | Non-palindromic; flipped alleles | Retain after sign test | Beta sign reversed, EAF equals 1 minus source EAF, OR reciprocal |
| 5, 6 | Palindromic; assume forward strand | Retain conditionally | Log shows strand consensus near 1; for MR and coloc, drop where EAF is near 0.5 |
| 7, 8 | Palindromic; assume reverse strand | Quarantine | Unexpected for a forward-consensus file |
| 1 to 4 | Palindromic; infer strand | Quarantine | Not in use per methods page; presence implies a different pipeline version |
| 9, 17, 18 | Palindromic dropped, or EAF missing or low | Exclude from effect-direction analyses | Record counts as drop reasons |
| 14, 16 | Required fields unknown; multiple reference matches | Exclude | Record counts; multiple matches are ambiguous |
| 15 | No matching variant in reference | Exclude | Dominant failure class in both example logs |
The documentation pages read here do not state whether rows with failure codes persist in the harmonised file or are removed. The pipeline should therefore reconcile the row count of the .h.tsv.gz against the log (mapped, harmonised and failed counts) and fail the ingest if they disagree. The final quality-control step also removes variants lacking a valid variant ID, chromosome, base pair location or p-value, so filtered counts can differ from mapped counts. Every drop reason needs its own denominator, as the audit below shows.
Metadata, licence, ancestry and sample overlap
Metadata is the contract that tells a pipeline what the numbers mean. The YAML sidecar sits next to each data file with the suffix -meta.yaml. The specification requires analysis software and version in the metadata so that p-values of zero can be interpreted ([58]), and the validation tool treats zero p-values as failures unless the analysis software is provided ([59]). An underflowed p-value should therefore be carried as a flagged value or as a negative log10 p-value, not coerced to the smallest representable number. The standard also allows EAF to be rounded up to a floor value stated in the metadata ([60]), which matters for rare-variant work.
Licence and terms must be stored per study. The methods page says newly submitted summary statistics have been required to be CC0 since 2022 ([61]), with special terms in rare cases appearing in an adjacent readme. The 2024 paper states a March 2021 cut-off for CC0 ([62]). Two dates therefore appear in current sources, and the discrepancy should be preserved in a data catalogue as "per study page". In the audited GCST90270927 directory the YAML sidecars carried no licence field, so terms had to come from the study page, not the file.
Ancestry and sample structure are not solved by metadata checks. Coloc's original model assumes the two datasets come from the same ethnic group with identical allele frequencies and LD patterns ([63]), whereas its documentation notes that coloc.abf does not assume similar LD structures provided a dense SNP map exists. Fine-mapping with a reference LD matrix needs a panel of similar ancestry ([13]) and consistent allele encodings. The LD reference therefore must be recorded and compared with sample ancestry.
Sample overlap needs its own gate. In two-sample MR, bias from sample overlap is a linear function of the proportion of overlap: with a null effect and a 10% one-sample relative bias, relative bias is 5% at 50% overlap and 3% at 30% overlap ([64]). A later study described overlap as a modifier of weak instrument bias and winner's curse ([65]). The sidecar fields documented on the methods page describe sample size and ancestry but not contributing cohorts, so overlap cannot be tested from the sidecar alone and needs a manual or cohort-registry step.
Beyond SNPs: CNV and gene-based data are different objects
The methods page says that since April 2026 CNV and gene-based GWAS data should follow the beyond-SNPs standards, and that previously these data were submitted without a standard schema, which limited downstream reuse ([66]). The linked landing page is more cautious: these are draft standards, not yet fully integrated into the submission system, and users are asked to validate data locally ([11]). It calls them proposed and evolving. This report preserves that tension. The methods page wording ("should follow") describes expectation, the landing page describes status (draft), and neither should be read as a finalised mandatory specification. The SNP harmoniser should not be assumed to process non-SNP input: the 2024 paper says files not harmonised include non-standard submissions such as CNVs or gene-based analyses ([67]), and the methods page says non-GWAS-SSF files can include gene-based, CNV and multivariate analyses where effect information is not available.
Table 4 compares field rules for the SNP standard and the two draft requirement pages as opened in October 2026. Draft cells should be re-checked at each release.
| Concept | SNP (GWAS-SSF) | CNV (draft) | Gene-based (draft) |
|---|---|---|---|
| Position | Chromosome and base pair location; build and 0/1-based in metadata | Chromosome, start and end, "0/1 based" | Chromosome, start and end of the gene |
| Identifier | Effect and other allele; optional variant ID and rsID | Region coordinates; no alleles | HGNC symbol or Ensembl gene ID, at least one |
| Effect size | One of beta, OR, HR; z-score fallback in v1.1.0 | At least one required: beta, OR, HR or z-score | Optional |
| Uncertainty | Standard error mandatory (#NA in z-score files) | Required with an effect size; z-score accepts none ([68]) | Required if an effect size is given |
| CI field names | ci_lower, ci_upper (encouraged) | confidence_interval_lower | ci_lower |
| Model or count | Not required | statistical_model_type required | n_snps optional |
| Minimum size | More than 100,000 SNPs | At least 10,000 pre-QC variants | At least 10,000 pre-QC genes |
One parser cannot serve all three. The CNV draft uses confidence_interval_lower while the gene draft uses ci_lower, and effect size is mandatory for CNV but optional for genes. The drafts also restrict combinations: odds ratio and hazard ratio are mutually exclusive in a file, and with several effect sizes one must be designated primary.
Gene-level statistics are produced differently from variant-level ones. MAGMA aggregates marker data to whole genes and reports a Z-value derived from the gene p-value ([69]). These are not effect-allele data and cannot enter variant-level colocalisation or MR without a separate, documented method.
Access routes: API status
The methods page says API access to summary statistics has been deprecated and that an improved API is coming soon ([70]). The Catalog API documentation states that REST API v2 is the only supported version ([71]), that the previous API was frozen in 2021 and retired in 2026, and that there is currently no API for summary statistics ([72]). The FAQ says the Catalog is working towards a new database for summary statistics. The wording differs on timing ("coming soon" against "working towards"), and neither page names a date. Bulk retrieval is by FTP, updated nightly, at paths based on the GCST accession, and a nightly list of harmonised studies is published.
- In the audited study, the accession directory holds the original data file, a YAML sidecar and a checksum file.
- Allele order is as submitted. GWAS-SSF recommends ALT as the effect allele and REF as the other allele where possible.
- Positions use a common genome build, and alleles are oriented to the forward strand where possible.
- In the audited study, the harmonised subdirectory holds the data file, its sidecar, a tabix index and a running log.
The legacy formatted file standardises headers and preserves the submitted data.
“An effect size is only meaningful relative to a named allele.
Implementation Considerations and Process Changes
Require a .h.tsv.gz file and a sidecar that declares is_harmonised true for effect-direction use.
Compare the genome assembly declared in the sidecar with the build of the reference panel.
Compute the downloaded file's md5 checksum and compare it with the sidecar value.
Match the file's row count to the running log's harmonised count plus any retained failure rows.
Recompute beta and effect allele frequency from the original file and verify the sign reversal and frequency adjustment.
For Mendelian randomisation and colocalisation, exclude palindromic variants with effect allele frequency near 0.5 unless orientation is verified.
An ingest pipeline for public summary statistics should be a set of explicit routes with acceptance tests, not one loader.
Four ingest routes
- SNP route (GWAS-SSF or pre-GWAS-SSF). Require a harmonised file and its sidecar. Refuse to substitute the original if harmonisation is pending. Record file name, md5 checksum, source URL, retrieval date, licence, genome assembly, harmonisation reference and pipeline version.
- CNV route (draft standard). Parse with a dedicated schema, validate locally as the landing page requests, store region coordinates rather than alleles, and keep these rows away from variant-level methods.
- Gene-based route (draft standard). Parse gene identifiers, p-value and optional effect fields. Join to variant-level evidence only through a documented gene-level step, never by position overlap alone.
- Non-standard route. Files flagged non-GWAS-SSF go to manual review because they may lack effect information.
Acceptance tests
Each test below is a proposal that is mechanical, produces a count, and can be written as code.
- State test. Sidecar says is_harmonised true and the file is a .h.tsv.gz; otherwise reject for effect-direction use.
- Build test. Sidecar genome_assembly equals the build of your reference panel (GRCh38 for current Catalog harmonised files).
- Checksum test. The md5 of the download matches the sidecar value.
- Row reconciliation. Rows in the .h.tsv.gz equal the log's harmonised count plus any retained failure rows.
- Flip test. For flipped codes (11, 13, 6), recompute beta and EAF from the original file and compare signs and 1 minus EAF.
- Palindromic test. Count codes 5, 6, 9, 17 and 18. For MR and colocalisation, exclude palindromic variants with EAF near 0.5 unless orientation is verified ([73]).
- Missing-field test. Count missing standard error and EAF; a missing standard error can be legitimate in z-score files, and #NA is the standard's missing-value token ([74]).
- p-value test. Flag p-values equal to 0 and read the analysis software from metadata. The validator also fails files with fewer than 100,000 rows by default ([75]).
- Reference test. Compare harmonised EAF with a reference panel for non-palindromic variants. Choose and document a deviation threshold against the reference population; frequency checks cannot detect problems near 0.5.
- Duplicate test. Decide a key (chromosome, position, ref, alt) and fail on duplicates; MungeSumstats checks duplicates by base pair position.
- Licence test. Store the licence per study and flag special terms.
Scientific sign-off gates
Metadata quality control cannot decide the following, so the pipeline should emit a review task for each:
- Sample overlap between exposure and outcome datasets; the direction and size of bias vary with the degree of overlap.
- Ancestry and LD reference. LD patterns can differ between ancestry groups.
- Colocalisation assumptions. The original model assumes at most one causal variant per trait in a region and two distinct datasets of unrelated individuals.
- Instrument strength and LD in drug-target MR. Report variance explained and the F statistic; unaccounted correlation between instruments can understate standard errors.
- Gene mapping. For target prioritisation, relative success estimates vary with confidence in variant-to-gene mapping.
Versioned refresh and source ledger
The sources moved during 2026: the specification reached v1.1.0 on 26 June ([9]), the validation tool published release 2.0.2 in September ([76]), and the harmoniser's latest release carried a missing-value fix ([77]). A refresh policy should therefore be tied to source releases, not calendar dates: re-run the acceptance tests and re-fetch sidecars when the specification, validator or harmoniser changes. Keep a ledger with, for each study, the immutable file names, URLs, retrieval dates, licence, pipeline version and test results.
Data Analysis and Evidence
A bounded real-study audit: GCST90270927
To anchor the checks in a real file, the accession directory GCST90270927 was opened on 19 November 2026. It holds the original data file, its YAML sidecar, a checksum file and a harmonised subdirectory. The harmonised subdirectory holds the .h.tsv.gz data file, its sidecar, a tabix index and a running log. The data file, index and log carry a modified time of 2023-11-03 ([78]), while the sidecars were modified on 2025-02-09. A pipeline should therefore store checksums and the log's pipeline version, not the sidecar date alone. The original sidecar declares GRCh37 and the harmonised sidecar GRCh38 ([25]).
Table 5 audits the log with fixed denominators. The release is the November 2023 harmonisation run, the target build is GRCh38 (source GRCh37), the reference is Ensembl release 95, and the pipeline version is 0.1.0. The log's percentages are of carried-forward sites. The input-based figures are calculated here from the log counts.
| Stage or class | Count | Share of carried-forward sites (log) | Share of input sites (calculated) |
|---|---|---|---|
| Input sites | 13,822,306 | n/a | 100% |
| Dropped, could not be mapped | 6,362 | n/a | 0.046% |
| Carried forward | 13,815,944 | 100% | 99.954% |
| Code 10, forward strand, correct | 6,382,570 | 46.20% | 46.18% |
| Code 11, forward strand, flipped | 4,854,469 | 35.14% | 35.12% |
| Code 5, palindromic, assumed forward, correct | 952,827 | 6.90% | 6.89% |
| Code 6, palindromic, assumed forward, flipped | 954,049 | 6.91% | 6.90% |
| Codes 12 and 13, reverse strand | 10,093 | 0.07% | 0.07% |
| Harmonised in total | 13,154,008 | 95.21% | 95.17% |
| Code 15, no matching reference variant | 661,899 | 4.79% | 4.79% |
| Code 16, multiple matching variants | 37 | 0.00% | 0.00% |
The log reports a palindromic mode of forward, inferred from a consensus on 10% of sites. Four readings follow. First, the harmonised fraction of input is 95.17% against the log's headline 95.21%, and the gap is the denominator difference between mapped and input sites; reporting only the headline hides the unmapped drops. Second, forward-strand, palindromic and reverse-strand flips total 5,812,386 variants, or 42.07% of carried-forward sites, so sign inversion is the rule, not the exception, for this file. Third, palindromic variants (codes 5 and 6) make up 13.80% of carried-forward sites and are oriented by assumption, so any MR or colocalisation using this file inherits that assumption. Fourth, failures are dominated by code 15, which accounts for 661,899 of 661,936 failed sites. This is a single file from a 2023 run with pipeline version 0.1.0 and must not be generalised.
Other quantified reference points
Aggregate figures give context without replacing a per-file audit. The Catalog's 2024 paper reports an average of about 3.5% unharmonisable variants removed ([7]), harmonisation of all GWAS-SSF files and 96% of pre-GWAS-SSF files, and 625,113 curated SNP-trait associations at July 2024. It also notes that the Catalog serves as a data feed to resources such as Open Targets Genetics.
Downstream platforms add their own filters. Open Targets ingests Catalog full summary statistics only when they have been harmonised ([79]), filters SNPs with unavailable beta, standard error or p-value ([80]), and flags curated lead variants with palindromic alleles. Its QC flags studies with fewer than 2,000,000 variants and makes them ineligible for SuSiE fine-mapping ([81]). For proteomics data it aligns allele order with gnomAD and flips the effect sign where alleles are reversed ([82]), and its locus-to-gene model is updated in each release, so scores are release-specific ([83]). Each downstream consumer thus repeats some harmonisation checks with its own thresholds, which argues for storing intermediate counts from your own pipeline.
Genetic-evidence magnitudes should be quoted with their limits. The 2015 analysis reported that selecting genetically supported targets could double the success rate in clinical development ([84]). The 2019 re-analysis noted that only 10% of candidate new molecular entities in early clinical trials are eventually approved. The 2024 analysis found relative success largely unaffected by effect size, minor allele frequency or year of discovery, and highest for Online Mendelian Inheritance in Man (OMIM) evidence at 3.7. None of these figures describes the risk that a file has the wrong effect direction.
“The scientific gates (sample overlap, ancestry compatibility, LD reference choice, instrument strength and colocalisation assumptions) remain human decisions, and no metadata field replaces them.
Implications and Future Directions
For data platform owners, file state is a first-class attribute. A data model should carry state, build, allele convention, harmonisation code, pipeline version and licence for every batch, and allow queries such as "only rows with codes 10 to 13". Harmonisation does not remove downstream checks: allele flips between z-scores and an LD reference are described as a common source of fine-mapping problems, so the check must be repeated when a different LD panel is introduced.
For statistical-genetics leads, the acceptance rules separate mechanical checks from scientific review. The mechanical checks can be automated and logged. The scientific gates (sample overlap, ancestry compatibility, LD reference choice, instrument strength and colocalisation assumptions) remain human decisions, and no metadata field replaces them. A defensible record states which gate was applied and who signed it off.
For tooling, three moving parts deserve watching. The specification is still evolving, as the 2026 z-score fallback shows ([9]). The draft non-SNP standards invite feedback, so a schema registry should version them and tolerate change. And the Catalog is developing a new summary statistics database, so retrieval code should sit behind an adapter. Alternative formats persist as well: GWAS-VCF stores one variant per row with the alternative allele as the effect allele, so an ingest layer supporting both conventions needs an explicit allele-convention field.
On open questions, no representative benchmark of harmonisation failure across the Catalog was found, and the threshold behind code 18 was not located in the documentation read. Teams that need either should measure them on their own refresh using the reconciliation tests. Scientific interpretation of any test result stays with the study team. Questions about licences and data use should go to the relevant legal professionals, as this report is not legal advice.
Frequently Asked Questions (FAQs)
What are GWAS summary statistics, and how are they used for drug discovery?
They are the full, aggregate association results for every variant analysed in a study. In drug discovery they feed colocalisation, MR and locus-to-gene scoring, where genetic support is associated with a higher probability of success for a mechanism. They support prioritisation, not proof of causality.
What does harmonising GWAS summary statistics mean?
In the Catalog it means reporting positions against the same and latest genome build, inferring missing chromosome, position and variant ID data where possible, and orientating alleles to the forward strand where possible. Variants that cannot be mapped or matched are removed or flagged.
What is the GWAS summary statistics format standard (GWAS-SSF)?
It is a two-file standard, a tab-separated data file and a YAML metadata file ([22]), in the Catalog since April 2023 and revised to v1.1.0 in June 2026 ([9]).
What are the Catalog's harmonised summary statistics, and where are they?
They are .h.tsv files in a harmonised subdirectory of each accession folder. A nightly text list of harmonised studies is on the FTP, and a folder with no harmonised file is likely awaiting harmonisation.
What quality control should I run on summary statistics?
Run the acceptance tests above: state, build, checksum, reconciliation, flip, palindromic, missing-field, p-value, reference frequency and duplicate checks. Published tools offer related checks, such as the GWAS-SSF validator's zero p-value rule ([59]).
Can I reuse public GWAS summary statistics in pharma analyses?
Often yes, subject to licence, which is per study. Newly submitted files are CC0, with special terms in rare cases, and older files may fall under standard EBI terms.
How does this relate to Mendelian randomisation for drug target validation?
MR requires the same allele for exposure and outcome, careful handling of palindromic variants, and attention to overlap and LD. External LD references are one way to model correlation between cis variants for drug targets.
How should CNV and gene-based GWAS data be handled?
Separately from SNP data. The beyond-SNPs standards are drafts ([11]) with different field rules for CNV and genes, and the SNP harmoniser should not be assumed to process them.
Conclusion
Public GWAS summary statistics can enter colocalisation, MR and target prioritisation, but only as files whose state, build, allele convention and exceptions are known. The Catalog documents original, formatted and harmonised files, a metadata sidecar, harmonisation codes and a running log, which together supply the raw material for an acceptance test suite. The most consequential invariant is effect direction. In the one real study audited, about four in ten mapped variants were flipped during harmonisation, so a pipeline that ignores file state would carry the wrong allele for a large share of effects.
Four limits remain. The standard is evolving, and a valid file may now lack a standard error. The CNV and gene-based formats are drafts with their own field rules, and the SNP harmoniser should not be assumed to cover them. Dropped records need their own denominators, and no representative failure benchmark is available, so each team should measure its own. Finally, metadata checks do not resolve sample overlap, ancestry or LD reference mismatch, which remain scientific decisions with named owners.
The practical recommendation is to treat each downloaded file as a versioned artefact with a ledger entry, run the mechanical tests on every refresh, and hold the scientific gates open until a statistical geneticist signs them off. Source status here is as of November 2026 and should be re-checked on each specification, validator or harmoniser release.
About IntuitionLabs
Build practical AI for pharma and biotech with IntuitionLabs. We help life-science teams turn complex information and workflows into useful software, governed knowledge systems and AI tools.
IntuitionLabs is an AI consulting, custom software development and data engineering firm serving pharmaceutical, biotechnology, medical-device and other life-science organizations. We work with clinical, regulatory, medical-affairs, commercial, quality and IT teams to connect technology decisions with the work people need to accomplish.
AI consulting and adoption
Our AI enablement services cover readiness assessments, use-case selection, governance and policies, team workshops, adoption measurement and ongoing advisory support. We help organizations structure the information layer behind AI: source material, context, permissions and maintained knowledge that make generated answers useful and reviewable. Private LLM inference and hosted AI options support teams evaluating how to operate AI with appropriate control over their data and infrastructure.
Software, data and life-science workflows
IntuitionLabs develops custom software for pharma and biotech, integrates enterprise systems, and builds data engineering and business intelligence solutions. Areas of focus include AI agents, regulatory research, medical writing, medical affairs, CMC information, competitive intelligence and clinical-document workflows. Our eTMF intelligence work includes cross-system reconciliation and inspection-readiness support.
Enterprise platforms and regulated delivery
We provide Veeva services, application support, managed services, integrations and custom applications, alongside enterprise content work involving platforms such as Egnyte. For regulated workflows, our services include GxP enablement, computer-system validation and software development addressing 21 CFR Part 11 requirements. The applicable controls, validation responsibilities and acceptance criteria are defined for each engagement.
Work with IntuitionLabs
Explore AI enablement, pharma and biotech software development, data engineering and BI, and Veeva services. Contact IntuitionLabs to discuss your workflow, information sources and implementation needs.
IntuitionLabs publishes educational research to help life-science teams make informed technology decisions. Coverage of a product or organization does not imply a client relationship, endorsement or partnership.
Sources / 84

Need Expert Guidance on This Topic?
Let's discuss how IntuitionLabs can help you navigate the challenges covered in this article.
I'm Adrien Laurent, Founder & CEO of IntuitionLabs. With 25+ years of experience in enterprise software development, I specialize in creating custom AI solutions for the pharmaceutical and life science industries.
The information contained in this document is provided for educational and informational purposes only. We make no representations or warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability, or availability of the information contained herein. Any reliance you place on such information is strictly at your own risk. In no event will IntuitionLabs.ai or its representatives be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from the use of information presented in this document. This document may contain content generated with the assistance of artificial intelligence technologies. AI-generated content may contain errors, omissions, or inaccuracies. Readers are advised to independently verify any critical information before acting upon it. All product names, logos, brands, trademarks, and registered trademarks mentioned in this document are the property of their respective owners. All company, product, and service names used in this document are for identification purposes only. Use of these names, logos, trademarks, and brands does not imply endorsement by the respective trademark holders. IntuitionLabs.ai is an AI software development company specializing in helping life-science companies implement and leverage artificial intelligence solutions. Founded in 2023 by Adrien Laurent and based in San Jose, California. This document does not constitute professional or legal advice. For specific guidance related to your business needs, please consult with appropriate qualified professionals.