Platform Documentation

Each type of analysis provides specialized metrics that help you assess the quality and reliability of results:

• Cluster Quality Evaluation: Ratings like "Good," "Very Good," or "OK" indicate how well-defined and separated your clusters are. These are composite scores based on a proprietary evaluation framework.
  • "Very Good" (0.8-1.0): Highly cohesive clusters with clear separation between groups
  • "Good" (0.6-0.79): Well-formed clusters with distinct boundaries and minimal overlap
  • "OK" (0.4-0.59): Recognizable clusters with some overlap between groups
  Higher quality clusters generally represent more meaningful IP groupings and are more likely to identify distinct infrastructure patterns.
• Stability Metrics (ARI/AMI): These scores (0-100%) measure how consistently the different clustering methods agree on the final IP grouping:
  • Adjusted Rand Index (ARI): Measures cluster assignment consistency across methods. 0.9+ indicates near-perfect agreement across methods, 0.7-0.9 suggests strong consensus with minor variations, and <0.5 indicates significant disagreement.
  • Adjusted Mutual Information (AMI): Information theory-based metric that quantifies shared information between different clustering results. Less sensitive to cluster size imbalance than ARI and particularly useful for identifying subtle pattern agreement.
  Higher scores indicate stronger consensus among methods, suggesting more reliable clustering results.
• Method Weights: These percentages show how much each clustering algorithm contributes to the final result. Methods with higher weights (e.g., 40%+) demonstrated better performance on your specific dataset and discovered particularly strong patterns, while methods with weights <10% generally detected weaker or less reliable patterns. These weights are dynamically calculated by the proprietary evaluation system and optimize the ensemble approach for your data.
• Confidence Scores: In prediction results, these percentages (0-100%) indicate how certain the model is about each IP's cluster assignment:
  • 90%+: Extremely reliable predictions with minimal ambiguity
  • 70-89%: Strong predictions with good reliability
  • 50-69%: Moderate confidence, may warrant additional verification
  • <50%: Low confidence, suggesting potential edge cases or ambiguous patterns
  Higher confidence (>80%) suggests more reliable predictions, while lower scores may warrant additional investigation.
• Prediction Kind (trust gate): Every prediction row carries a kind discriminator that is the primary signal a SOC / SOAR consumer should key on:
  • confident_match: Strong fingerprint match. The model is confident in the cluster attribution. Recommended actionable subset for correlation rules.
  • ambiguous_diffuse: Top-1 leans, but probability mass is spread thin across many candidates. Treat the attribution with softer trust and review the candidate set.
  • ambiguous_split: Close two-way / few-way tie between candidates. Investigate the candidate set as a unit; the true match is likely one of them.
  • out_of_distribution: No fingerprint match. The IP does not resemble any trained cluster. predicted_cluster and confidence are intentionally null on these rows; investigate via behavioral evidence rather than the cluster attribution.
• Candidates set: Each row carries the top-K candidate clusters above an entropy-aware confidence floor. For ambiguous rows iterate the set as a whole rather than relying on top-1 alone.
• top1_minus_top2: The gap between the leading candidate and the runner-up. Large gaps mean a clean win; small gaps indicate ties and pair with ambiguous_split.
• effective_n: exp(entropy) over the candidate distribution, interpretable as the "number of candidates worth of probability mass." Near 1.0 ⇒ confident, large values ⇒ diffuse hedging.
• Feature Importance Rankings: Shows which characteristics most strongly define each cluster, with higher-ranked features having greater influence on cluster formation. This helps security analysts understand the primary attributes driving IP groupings.
• ROC AUC (Receiver Operating Characteristic Area Under Curve): Measures discrimination ability across classification thresholds (0.5-1.0). 0.5 indicates no better than random guessing, 0.7-0.8 shows acceptable discrimination, 0.8-0.9 demonstrates excellent discrimination, and 0.9+ signals outstanding discrimination ability. This metric is particularly valuable for comparing different models' overall performance.
• Log Loss: Measures prediction uncertainty by heavily penalizing confident but wrong predictions. Lower values indicate better calibrated probability estimates, which matters for risk assessment.
• Class-specific Metrics: Performance measurements broken down by individual clusters show which types of IP groups the model handles well or poorly. This helps focus refinement efforts.
• Actor Labels: User-generated rule-based identifiers that flag potential threat patterns based on learned cluster characteristics. They help analysts recognize and categorize suspicious activity quickly, without manual pattern matching.

These metrics form the basis for calculating precision, recall, F1 score, and other performance metrics:

• Precision = TP / (TP + FP): The proportion of IPs assigned to a cluster that actually belong there. Higher precision means fewer false positives.
• Recall = TP / (TP + FN): The proportion of IPs that actually belong to a cluster that were correctly assigned there. Higher recall means fewer false negatives.
• F1 Score = 2 × (Precision × Recall) / (Precision + Recall): The harmonic mean of precision and recall, providing a balance between the two metrics.

In security analysis, these metrics have specific implications:

• High precision ensures that IPs flagged as part of a potential threat cluster genuinely share characteristics, reducing false alarms.
• High recall ensures that you capture most IPs belonging to a specific threat pattern, minimizing missed detections.
• Balanced F1 score indicates a model that effectively identifies threat infrastructure without generating excessive false positives or missing critical indicators.