Introduction

In fancy color it is especially easy to confuse impression with measurement. “Bright”, “rare”, “expensive” are good words for a sales conversation and poor features for a model. Intensity is tied to hue, saturation, lightness, cut and the light in which the stone is shown. A laboratory builds its own procedure for this; commercial data almost always live by less strict rules.

So machine learning should not be presented as a new way of certification. Its more modest, but useful, role is to unpack regularities in a known labelling, find disputed records, and check where instrumental indicators actually carry signal. In that mode the model is an analyst’s working instrument, not an automatic gemmologist.

The note compares two versions. V1 trained on 153 observations. In the minimal setting it saw CHROMA; in the full setting, two further human labels. V2 is built on 226 cleaned records from open Cutwise projects and receives scanner colorimetry plus stone shape. The second version is closer to a pre-screen, because the operator need not name a class before the model input. That does not make it a substitute for a report.

Accuracy in this report is accuracy of predicting the target label in a particular sample and under a chosen split scheme. It is not the share of “correctly identified GIA stones” in the whole world. For rare classes and neighbouring grades that distinction decides almost everything.

Method and sources

The work joins two experimental contours. In V1 the classes are ordered Y-Z, Light, Fancy, Intense, Vivid. The original size is n=153. Features include CHROMA, and in the full configuration ЦВЕТ_ПРОГНОЗ and ЦВЕТ_ТАБЛИЦА — results of human marking. Random Forest and Explainable Boosting Machine (EBM) were compared; quality was assessed by cross-validation.

Before the V2 calculation the data were cleaned, after which n=226 cleaned remained. The set includes Cushion, Pear, Oval, Radiant. The model receives LCH coordinates, Munsell, Cutwise Color and shape. A histogram built in the working process came out empty. That is a visualisation defect, not evidence of a uniform or “good” data distribution. Class distribution is therefore taken from a direct count of rows.

For V2, CatBoost, XGBoost, Decision Tree, Random Forest and logistic regression were tested. Beside accuracy, mean absolute rank error, MAE, is given. Intensity classes are ordered: a miss to a neighbouring step is not equal to a jump of two or three. MAE notices that, though it does not speak of the commercial cost of the error.

Technological context comes from OctoNus materials on fancy-color optimisation and the Cutwise platform. Scientific background is King et al., 1994, Gems & Gemology, 30(4):220–242. GIA in the title and discussion denotes the familiar language of intensity; GIA training labels are not assumed unless that is said of the set itself.

What is predicted, and what is not

The target variable here is a discrete intensity class already present or assigned in the source data. It is not a direct physical colour measurement and not a money valuation. The models do not determine price, origin, clarity, polish, certificate quality or investment appeal. And they do not promise that two stones with close coordinates will look the same in any light.

The order of classes matters. Light is closer to Fancy than to Vivid; Intense usually neighbours Fancy and Vivid, not Light. A classifier that mostly errs to a neighbouring step may be more useful than a model with the same accuracy but rare distant misses. In a sale even a neighbouring error is sometimes painful: Intense and Vivid create different expectations in the buyer.

Two tasks must be separated. The first is to predict an already known labelling in a documented dataset. The second is independently to establish the colour grade of a specific stone. The report belongs to the first. The second requires a standardised procedure, an instrument, observation and the laboratory’s responsibility.

V1: sample, classes and CHROMA

V1 has n=153 records. The CHROMA range is about ≈37.0–92.8. Classes are uneven: Y-Z — 19, Light — 26, Fancy — 76, Intense — 25, Vivid — 7. Fancy is almost eleven times more numerous than Vivid. In such a structure overall accuracy easily looks better than the real position of rare classes.

A baseline Random Forest given only CHROMA obtained 64.7% accuracy and F1macro 0.622. The result is substantive. Saturation does help distinguish intensity, but one numeric feature does not hold the boundaries of the whole scale. Similar values can appear for different hues, shapes, proportions and measurement conditions.

The full Random Forest reached 69.9% accuracy and F1 0.698. EBM on the full set — 76.5% accuracy and F1 0.742. In the interpretable model, CHROMA importance is about ≈1.62. That is not a contribution percentage and not a physical constant. It is importance inside a particular model, assembled on a particular representation of the data.

The main nuance of V1 hides precisely in the rise of the result. Full models see ЦВЕТ_ПРОГНОЗ and ЦВЕТ_ТАБЛИЦА — a person’s scores. Such an algorithm cannot honestly be offered as a way to assign a grade before hand labelling: the expert signal has already entered the input. It is fit, though, for checking consistency, comparing two labellings, finding unusual records and an internal audit of decisions.

The term “leakage” should not be hidden behind a technical wording. This is not necessarily a bug in the code. Sometimes the model honestly receives a column that, in the real process, is not yet there at that moment. Then the quality score answers “how well the algorithm repeats an expert decision already begun”, not “what it will do before the expert”. For internal control that may be exactly the needed task. For a promise of automatic grade — no longer.

V1: class comparison and the cost of imbalance

Macro-F1 puts each class on equal terms and is therefore more informative here than a single overall accuracy. Below — per-class F1 for three settings: Random Forest with CHROMA only, full Random Forest, and EBM.

The most alarming signal in the table is 0.261 for INTENSE in the CHROMA-only model. That is not a side detail. Intense sits at the join of Fancy and Vivid, so it is vulnerable to measurement noise, unclear labelling and a small number of examples. With extra features F1 rises to 0.480 for Full RF and 0.654 for EBM. But that improvement cannot be separated from the fact that part of the new features already express a human decision.

Vivid’s F1 looks high with only seven observations. That does not add confidence: a few examples in a test fold can shift the indicator sharply. LIGHT, by contrast, almost does not gain from added features — 0.583, 0.565, 0.571. In an applied system after such a result it is more useful to show not only a label, but a degree of confidence, nearest alternatives and a reason to send the case to a hand check.

V2: the move to scanner measurements

V2 asks a different question: can a working prediction be obtained without a labeller on the input? Here n=226 cleaned records of Cushion, Pear, Oval and Radiant remain. Instead of hand fields, LCH lightness and chroma, Munsell, Cutwise Color and shape are used. With equally standardised measurement this feature set can in principle be obtained again — before a preliminary opinion of the operator.

V2 target classes distributed as follows: Intense 87 (38.5%), Vivid 50 (22.1%), Fancy 47 (20.8%), Deep 24 (10.6%), Light 18 (8.0%). Deep was absent in V1. For that reason V1 and V2 cannot be read as two measurements of one scale and the second version declared “better by so many points”. These are other inputs, another sample and another class composition.

An empty histogram says nothing about bimodality, outliers, or the shape of continuous-feature distributions. It is more correct to record the chart failure and lean on the class table. On a repeat of the work, visualisation should be fixed separately, not replaced with assumptions about the data.

V2 correlations and the risk of duplicate features

The following Pearson correlations with the target structure or intensity signal were recorded: chroma/lightness ratio 0.819, cutwise_color_chroma 0.805, lch_chroma 0.785, lch_lightness −0.492. A positive link for saturation and its ratio to lightness matches the expectation: a stronger colour signal in this set more often goes with higher intensity. LCH lightness moves the other way.

Correlation does not thereby become a causal explanation. Several columns may describe a close physical property with different formulae. For a linear model this is a multicollinearity risk; for a tree and boosting — a chance to lean on duplicate signals and then transfer worse to another instrument. Importance analysis in such a situation is easy to over-read.

Shape in the model is not a table decoration. Cushion, Pear, Oval and Radiant work differently with cut, light distribution and the commercial description of colour. But shape cannot be allowed to become an indirect label. Validation should check whether the algorithm is memorising small groups or a data source instead of a stable regularity.

V2: models and cross-validation

On CV, V2 was led by CatBoost: 85.93% accuracy and rank MAE 0.208. XGBoost obtained 81.64% / 0.278, Decision Tree 79.97% / 0.307, Random Forest 77.66% / 0.353, logistic regression 73.02% / 0.443.

The gap between CatBoost and logistic regression looks like a trace of non-linear boundaries and interactions among colour features and shape. But one cannot conclude from it that CatBoost “understands colour”. The model better separates the labels of this set under this cross-validation scheme. The next check should include a held-out batch, results by shape, confusion matrices and probability calibration.

MAE 0.208 means a small average error in the encoded ordinal space. It does not assign a cost to a miss. If a business rule does not allow Vivid to be named instead of Intense, separate thresholds, probability analysis and a refusal mechanism will be needed. A single confident-looking label makes the risk invisible.

Here a practical fork begins. The model can be forced to answer every request — and a handsome summary metric obtained. Or a range can be set in advance where it does not choose a class and returns the record to a specialist. The second path will reduce the share of automatically processed stones, but will not mask borderline cases. For small data that is usually more honest than imitating a faultless automaton.

OctoNus, Cutwise and the bounds of the technical layer

The OctoNus page on fancy-color diamonds sets the technological context of optimisation work with coloured stone. Cutwise is the source of open project data and measurement representations used in V2. In that sense the model is interesting as a bridge between scanner and analytics: its features can be obtained before a hand classification comment.

It does not follow that there is official certification from OctoNus, Cutwise or GIA. A stone is not like a flat colour sample. Cut, orientation, reflections, fluorescence, lighting, instrument setup and sample preparation all affect the result. A scanner gives a repeatable signal inside its measurement chain; transferring readings to another instrument without calibration cannot be treated as automatic.

King et al. (1994) is useful precisely as this reminder: fancy-color terminology ties observation, comparison and systematic description, and does not reduce to one universal sensor. On a product card a model label cannot be placed next to a laboratory certificate as if both documents were equal in status.

Synthesis and discussion

V1 showed that in a small set CHROMA and extra features contain a strong classification signal: from 64.7% for CHROMA-only RF to 76.5% for EBM. The rise, however, does not mean autonomous automation. Full models receive human labels on the input. Intense proved the most vulnerable place, and Vivid is too small for broad confident conclusions.

V2 removes exactly that input problem. CatBoost sees scanner indicators and shape, not a labeller’s opinion, and at n=226 cleaned obtains 85.93% CV accuracy and MAE 0.208. This is not “a 9.43-point improvement on V1”: classes, data and task setting all differ.

The overall conclusion is more modest — and more useful than a loud formula. Measurable colour features allow working classifiers to be built in limited data. But inputs, split, imbalance and zones of uncertainty need to be shown honestly. A normal product interface here is a hint, a probability, neighbouring classes and a hand-off of disputed records to a person.

The question remains how to embed this conclusion in a real process. In sorting, a batch may first pass a scanner, then a model ranking layer, and only then a hand check of low-confidence cases. On a product card a model score is better kept as an internal technical attribute, not issued as part of a laboratory document. That order does not make processing “objective” in a philosophical sense, but it leaves an auditable chain: measurement, prediction, human decision.

Scenarios and hypotheses

Limitations

The samples are small: 153 records in V1 and 226 after cleaning in V2. Classes are unbalanced: Vivid V1 includes seven observations, Light V2 eighteen. Cross-validation reduces dependence on a single split, but does not replace an external test and does not confirm transfer to other batches, colours, shapes, laboratories or instrument settings.

In V1 there is a risk of meaning leakage through hand features. ЦВЕТ_ПРОГНОЗ and ЦВЕТ_ТАБЛИЦА help predict the label because they are already tied to an expert decision. V2 avoids this particular risk, but depends on cleaning, availability of scanner values and the set of shapes. An empty histogram gives no visual grounds to judge continuous features.

Neither V1 nor V2 replaces GIA. They do not establish an official colour grade, do not issue a certificate and do not determine value. For a sale, an insurance document and a public product card it is important to distinguish a model score, an internal screen and a laboratory report.

Conclusions

1. V1 at n=153 showed the informativeness of CHROMA, but its full models depend on human labels on the input.
2. V1: RF CHROMA only — 64.7% and F1macro 0.622; Full RF — 69.9% and F1 0.698; EBM — 76.5% and F1 0.742.
3. Intense remains V1’s zone of uncertainty; rare Vivid requires a cautious reading of metrics.
4. V2 at n=226 cleaned uses scanner colorimetry and shape; CatBoost leads with 85.93% CV accuracy and MAE 0.208.
5. The most realistic use of the models is research, triage and quality control with a person involved, not a replacement for GIA.

FAQ

Can fancy-color intensity be predicted from a scanner? In V2 a combination of scanner features and shape gave CatBoost 85.93% accuracy on CV. That is a result of a particular set, not a guarantee for any stone or instrument.

Why can V1 not be called fully automatic? Full V1 uses ЦВЕТ_ПРОГНОЗ and ЦВЕТ_ТАБЛИЦА, that is human labels. A person already takes part on the model input.

Does the model replace a GIA report? No. It is an internal research class prediction, not a laboratory grade, certificate or valuation.

What does MAE 0.208 mean? It is mean absolute error on the numeric order of classes. The metric complements accuracy and shows how far, on average, the prediction sat from the true grade.

Why is Intense harder to predict? In V1 the class sits between neighbouring grades and showed F1 from 0.261 to 0.654 in different models. Extra borderline examples and an error matrix are needed.

Bibliography

1. OctoNus. Fancy-color diamonds: https://www.octonus.com/projects/fancy-color-diamonds
2. Cutwise. Public projects and platform description: https://cutwise.com/pr/
3. King, J. M., Moses, T. M., Shigley, J. E., Welbourn, C. M., Lawson, S. C., Cooper, M. 1994. Characterizing Natural-Color Type IIb Blue Diamonds. Gems & Gemology, 30(4), 220–242.
4. BrillPulse. Experimental datasets and V1/V2 notebooks used for the stated metrics.
5. GIA. Public materials on terminology and laboratory assessment of coloured diamonds; used as context, not as a certificate of the model.