Checking the implementation
Comparing independent calculations on the same model tests whether the methods give consistent results. Agreement does not establish that the model describes every real material.
THE SCIENCE BEHIND STRUCTSUB
Evidence behind
Evidence behind
the results.
StructSub's calculations set against measured data and against published reference calculations run on the same model. Every number on this page is produced by a script in the StructSub repository and compiled from its stored output, so each one can be reproduced, and limitations are reported as found.
Published referencesMethods & conditionsReported limitations
Using these results. When a result here supports published work, cite StructSub (see How to cite) together with the reference listed for that comparison. Comparisons with a reference calculation test the implementation on an identical model; comparisons with measurement test the model itself.
What each comparison tells you
Checking the physical model
Comparison with experiment tests how well the chosen model represents the measured system. Structure, temperature, loading and other conditions matter when interpreting differences.
- Uncertainty & difference
- The methods below explain the reported errors. For adsorption, ± is one standard error and the difference is expressed in combined standard errors (σ).
- Units & conditions
- Read each table alongside its temperature, pressure and method. Values from different conditions or models are not interchangeable.
- Scope & limitations
- These benchmarks describe the systems tested. Each section retains its findings and limitations, including cases where agreement is poor.
Adsorption isotherms
Grand canonical Monte Carlo in ten systems chosen to stress different parts of the method: narrow channels, a skewed cell, a zeolite, a charged quadrupolar guest, a binary mixture and argon at 87 K. Each was run with this package and with an independent published code on an identical model, so a difference is a difference between implementations, not force fields.
| Framework | Guest | T (K) | p (bar) | Loading (mol/kg) | Reference | Difference (σ) | Heat (kJ/mol) | Reference |
|---|---|---|---|---|---|---|---|---|
| IRMOF-1 | CO₂ | 298 | 10 | 13.51 ± 0.31 | 13.71 ± 0.24 | 0.5 | 18.9 ± 0.4 | 19.0 ± 0.2 |
| ZIF-8 | CH₄ | 298 | 10 | 4.85 ± 0.03 | 4.87 ± 0.04 | 0.5 | 17.9 ± 0.1 | 17.9 ± 0.1 |
| ZIF-8 | CO₂ | 298 | 1 | 1.89 ± 0.03 | 1.83 ± 0.03 | 1.4 | 20.0 ± 0.2 | 20.1 ± 0.2 |
| MIL-47 | CO₂ | 298 | 1 | 4.81 ± 0.07 | 4.87 ± 0.11 | 0.4 | 26.5 ± 0.2 | 26.5 ± 0.1 |
| MIL-47 | CO₂ | 298 | 20 | 11.43 ± 0.08 | 11.45 ± 0.20 | 0.1 | 34.1 ± 0.3 | 33.8 ± 0.8 |
| MIL-47-skew | CO₂ | 298 | 1 | 4.82 ± 0.09 | 4.91 ± 0.07 | 0.8 | 26.5 ± 0.3 | 26.3 ± 0.4 |
| MFI | CH₄ | 298 | 1 | 2.89 ± 0.01 | 2.90 ± 0.02 | 0.7 | 31.7 ± 0.5 | 31.7 ± 0.8 |
| MIL-47 | N₂ | 298 | 10 | 2.63 ± 0.03 | 2.69 ± 0.03 | 1.6 | 14.2 ± 0.0 | 14.1 ± 0.2 |
| MIL-47 | CO₂ in CO₂ / N₂, 50:50 | 298 | 1 | 2.39 ± 0.07 | 2.32 ± 0.06 | 0.8 | ||
| MIL-47 | N₂ in CO₂ / N₂, 50:50 | 298 | 1 | 0.159 ± 0.003 | 0.167 ± 0.003 | 1.8 | ||
| IRMOF-1 | Ar | 87 | 0.001 | 1.31 ± 0.05 | 1.35 ± 0.04 | 0.6 | 9.6 ± 0.1 | 9.5 ± 0.1 |
Method & conditions
UFF framework parameters with published guest models, Lorentz-Berthelot mixing, a 12 Å cut-off with no tail correction, DDEC framework charges where charged, the same supercell and the same Peng-Robinson fugacities on both sides. Errors are one standard error; the last loading column is the difference in combined standard errors. Heats of adsorption are in kJ/mol.
Reproducibility details
Repository command for this comparison:
validation/reference_gcmc/suite.py (needs the reference code installed)Henry coefficients and heats at zero loading
Widom insertion in MOF-5 (EDUSIF) at 300 K, which integrates the whole host-guest energy surface rather than one pose. The last two rows drive the shipped Henry routine itself, so its statistics and its enthalpy formula are checked, not only the energy beneath it.
| Quantity | This package | Reference | Agreement |
|---|---|---|---|
| Widom Rosenbluth weight, Lennard-Jones only | 22.287 ± 0.087 | 22.305 ± 0.120 | 0.08%, 0.1σ |
| Excess chemical potential | −7.7424 kJ/mol | −7.7444 kJ/mol | 0.002 kJ/mol |
| Widom weight with framework charges (Ewald) | 5.096 ± 0.050 | 4.952 ± 0.193 | 2.8%, 0.7σ |
| Excess chemical potential, zero loading | −7.7387 kJ/mol | −7.7353 ± 0.0278 kJ/mol | 0.0034 kJ/mol, 0.1σ |
| Enthalpy of adsorption, zero loading | −15.634 ± 0.088 kJ/mol | −15.676 ± 0.393 kJ/mol | 0.042 kJ/mol, 0.1σ |
Method & conditions
One Lennard-Jones probe with equalσ on every framework site, so the two mixing rules coincide exactly; 12 Å hard cut-off. Stored in test/test_data/raspa_widom_reference.json and re-checked by the test suite.
Reproducibility details
Repository command for this comparison:
validation/reference_gcmc/lennard_jones.py, electrostatics.py, zero_loading.pyGuest diffusion
Self-diffusivity by molecular dynamics in IRMOF-1 at 298 K, against the same model run with the reference code. The carbon dioxide case includes rotation and Ewald electrostatics with DDEC charges.
| Guest | This package (10⁻⁸ m²/s) | Reference (10⁻⁸ m²/s) | Difference |
|---|---|---|---|
| CH₄ | 2.98 ± 0.09 | 2.96 | 0.7% |
| CO₂ (DDEC charges) | 1.450 ± 0.038 | 1.428 | 1.5% |
Method & conditions
Rigid guests in a rigid framework; several independent trajectories, each from its own GCMC snapshot; diffusivity from the long-time slope of the mean squared displacement, fitted the same way on both sides. Recorded 28 September 2026.
Reproducibility details
Repository command for this comparison:
validation/reference_gcmc/diffusion.pyPartial atomic charges
Charges from the graph neural network and from charge equilibration (QEq), compared atom by atom with DDEC charges from periodic DFT. The network was trained on QMOF DDEC charges; on held-out frameworks its mean absolute error is 0.0044 e (0.013 e on metals). The four frameworks below are independent of its training set.
| Framework | Atoms | Network MAE (e) | QEq MAE (e) | Metal charge: network / QEq / DDEC (e) |
|---|---|---|---|---|
| IRMOF-1 | 106 | 0.048 | 0.193 | Zn: +0.97 / +0.39 / +1.10 |
| ZIF-8 | 276 | 0.034 | 0.167 | Zn: +0.64 / +0.12 / +0.76 |
| MIL-47 | 72 | 0.058 | 0.183 | V: +1.81 / +1.25 / +2.01 |
| MIL-53(Cr), lt | 38 | 0.068 | 0.179 | Cr: +1.54 / +1.37 / +1.76 |
Method & conditions
Mean absolute error per atom against DDEC (PBE). Over 2,670 CoRE MOF frameworks QEq correlates with DDEC (r = 0.89) but underestimates its magnitude (best-fit slope 0.56), which is why the network is the default. Experimental hydrogen positions are normalised to the training bond lengths before prediction.
Reproducibility details
Repository command for this comparison:
validation/charges.pyBattery materials
The average intercalation voltage of LiFePO₄ against lithium metal, and the lithium migration barrier of the rate-limiting hop in a cathode, a sulfide solid electrolyte and a fast conductor. Measured: LiFePO₄ at 3.5 V; Li₇P₃S₁₁ at 1.7 × 10⁻² S/cm with an activation energy of 17 kJ/mol (0.18 eV).
| Model | Voltage (V) | Reference (V) | Capacity (mAh/g) | Volume change (%) |
|---|---|---|---|---|
| MACE-MP-0b3 | 3.45 | 3.5 (measured) | 170 | +5.4 |
| MACE-MPA-0 | 3.46 | 3.5 (measured) | 170 | +4.9 |
| Material | Fast (eV) | Accurate (eV) | Reference (eV) |
|---|---|---|---|
| LiFePO₄ | 0.35 | 0.30 | 0.27 (DFT) |
| Li₇P₃S₁₁ | 0.13 | 0.16 | 0.18 (measured, Ea) |
| Li₃N | 0.15 | 0.01 |
Method & conditions
Voltage: LiFePO₄, FePO₄ and lithium metal each relaxed, atoms and cell, with the same model; theoretical capacity 170 mAh/g. Barriers: climbing-image nudged elastic band, with the bond-valence force field (Fast) and MACE-MP-0b3 (Accurate). GFN2-xTB is not offered for voltages: its self-consistent field did not converge on LiFePO₄, whose high-spin iron it does not describe. Li₃N has no reference barrier here; the Fast model is weakest on such near-free hops, so use the Accurate value for it. Structures from the Crystallography Open Database (CC0).
Reproducibility details
Repository command for this comparison:
validation/batteries.pyElastic constants
ZIF-8 single-crystal elastic constants against Brillouin scattering. The default force field is 1.4 to 5.4 times too stiff, the shear constant worst: use it to compare frameworks with one another, not to quote a modulus.
| Constant | UFF4MOF (GPa) | Measured (GPa) | Ratio |
|---|---|---|---|
| C₁₁ | 21.8 | 9.52 | 2.3× |
| C₁₂ | 9.9 | 6.87 | 1.4× |
| C₄₄ | 5.3 | 0.97 | 5.4× |
Method & conditions
Relaxed-ion stiffness from finite strains of the relaxed cell, cubic constants averaged over equivalent directions. CoRE MOF 2014 ZIF-8 geometry.
Reproducibility details
Repository command for this comparison:
validation/mechanics.pyReferences
- [1] A. K. Padhi, K. S. Nanjundaswamy and J. B. Goodenough, Phospho-olivines as positive-electrode materials for rechargeable lithium batteries, J. Electrochem. Soc. 144, 1188 (1997). doi:10.1149/1.1837571
- [2] A. S. Rosen et al., Machine learning the quantum-chemical properties of metal-organic frameworks for accelerated materials discovery, Matter 4, 1578 (2021). doi:10.1016/j.matt.2021.02.015
- [3] D. Morgan, A. Van der Ven and G. Ceder, Li conductivity in LixMPO4 (M = Mn, Fe, Co, Ni) olivine materials, Electrochem. Solid-State Lett. 7, A30 (2004). doi:10.1149/1.1633511
- [4] D. Nazarian, J. S. Camp and D. S. Sholl, A comprehensive set of high-quality point charges for simulations of metal-organic frameworks, Chem. Mater. 28, 785 (2016). Data: CoRE MOF 2014 DDEC database, doi:10.5281/zenodo.3986573 (CC-BY-4.0). doi:10.1021/acs.chemmater.5b03836
- [5] J.-C. Tan, B. Civalleri, C.-C. Lin, L. Valenzano, R. Galvelis, P.-F. Chen, T. D. Bennett, C. Mellot-Draznieks, C. M. Zicovich-Wilson and A. K. Cheetham, Exceptionally low shear modulus in a prototypical imidazole-based metal-organic framework, Phys. Rev. Lett. 108, 095502 (2012). doi:10.1103/PhysRevLett.108.095502
- [6] Y. A. Ran, S. Sharma, S. R. G. Balestra, Z. Li, S. Calero, T. J. H. Vlugt, R. Q. Snurr and D. Dubbeldam, RASPA3: A Monte Carlo code for computing adsorption and diffusion in nanoporous materials and thermodynamics properties of fluids, J. Chem. Phys. 161, 114106 (2024). doi:10.1063/5.0226249
- [7] Y. Seino, T. Ota, K. Takada, A. Hayashi and M. Tatsumisago, A sulphide lithium super ion conductor is superior to liquid ion conductors for use in rechargeable batteries, Energy Environ. Sci. 7, 627 (2014). doi:10.1039/C3EE41655K