Features Built for Modern QA Teams
Everything you need to automate, analyze, and accelerate your testing workflow. From flexible pipelines to intelligent AI analytics — TestHide has you covered.
8 Dynamic AI Models Working for You
TestHide's AI ecosystem learns from your project data to provide intelligent classification, prediction, and insights — all trainable and configurable through the UI.
Root-Cause Classifier
Multi-modal neural network that analyzes test failures to automatically classify them into categories like "Environment Issue", "Test Bug", "Product Bug", "Flaky", etc.
Flakiness Predictor
Gradient boosting model that predicts the probability of a test failing on the next run, based on historical patterns and code metrics.
Failure Retriever
Vector similarity search that finds past failures similar to the current one, helping engineers quickly find known solutions.
Novelty Detector (OOD)
Out-of-Distribution detector that flags when a failure pattern is significantly different from anything seen during training.
Log Signature Miner
Template extraction that identifies recurring log patterns across failures, enabling signature-based classification without neural networks.
Visual Diff Analyzer
Screenshot comparison that detects visual regressions in UI tests, going beyond pixel-diff to understand semantic changes.
Bug Linker
Automatic linking of test failures to existing Jira tickets based on semantic similarity and historical associations.
Emerging Issues Detector
Time-series analysis that identifies sudden spikes in failure patterns, alerting teams to emerging problems before they escalate.
AI Model Dashboard
Train, monitor, and deploy models from the UI
Deep Dive: AI Model Capabilities
Each model is designed to solve specific QA challenges. Click to explore benefits and technical details.
Root-Cause Classifier
Multi-modal neural network that analyzes test failures to automatically classify them into categories like "Environment Issue", "Test Bug", "Product Bug", "Flaky", etc.
Key Benefits:
- Reduces manual triage time by 80%
- Learns from your team's historical classifications
- Provides confidence scores and top-5 predictions
- Supports text, metrics, and image embeddings
Technical Details
Transformer-based text encoder (BERT) with numeric projections and optional image vector branches. Temperature-calibrated softmax for reliable probability estimates.
Flakiness Predictor
Gradient boosting model that predicts the probability of a test failing on the next run, based on historical patterns and code metrics.
Key Benefits:
- Identify risky tests before they break CI
- Feature importance shows WHY tests are flaky
- Configurable threshold for quarantine decisions
- Works with any test framework
Technical Details
LightGBM classifier trained on 50+ features including log dynamics, test history, git churn, and runtime metrics. Calibrated with isotonic regression.
Failure Retriever
Vector similarity search that finds past failures similar to the current one, helping engineers quickly find known solutions.
Key Benefits:
- Instantly find similar past failures
- Link to existing fixes and workarounds
- Learn from team knowledge base
- Sub-second latency with FAISS indexing
Technical Details
Dual-encoder architecture with contrastive learning. Embeddings stored in FAISS IVF-PQ index for efficient approximate nearest neighbor search.
Novelty Detector (OOD)
Out-of-Distribution detector that flags when a failure pattern is significantly different from anything seen during training.
Key Benefits:
- Catch truly novel issues immediately
- Avoid false confidence on unknown patterns
- Trigger human review for anomalies
- Improve model reliability over time
Technical Details
Hybrid approach: Tiny Autoencoder reconstruction error + Isolation Forest on embedding space. Fusion score with configurable threshold.
Log Signature Miner
Template extraction that identifies recurring log patterns across failures, enabling signature-based classification without neural networks.
Key Benefits:
- Deterministic, interpretable classifications
- Zero-shot categorization for new errors
- Template rules exportable as CSV
- Complements neural classifiers
Technical Details
Drain3-based log parsing with custom tokenization. Generates signature rules with per-class probability distributions.
Visual Diff Analyzer
Screenshot comparison that detects visual regressions in UI tests, going beyond pixel-diff to understand semantic changes.
Key Benefits:
- Detect UI regressions automatically
- Classify diff types (text, button, layout)
- Ignore dynamic content areas
- Generate visual diff previews
Technical Details
ResNet-50 backbone with SSIM loss. Semantic segmentation to identify changed regions and their types.
Bug Linker
Automatic linking of test failures to existing Jira tickets based on semantic similarity and historical associations.
Key Benefits:
- Auto-link failures to known bugs
- Reduce duplicate ticket creation
- Surface relevant context instantly
- Learn from manual links over time
Technical Details
Dense retrieval over Jira ticket embeddings. Combines title, description, and historical test-to-issue mappings.
Emerging Issues Detector
Time-series analysis that identifies sudden spikes in failure patterns, alerting teams to emerging problems before they escalate.
Key Benefits:
- Early warning for infrastructure issues
- Detect deployment-related failures
- Track failure velocity trends
- Automatic alerting thresholds
Technical Details
Rolling window statistics with Z-score anomaly detection. Grouped by error signature for pattern-level alerting.
7 Powerful Build Step Types
Mix and match step types to create pipelines that fit your exact workflow. From simple shell scripts to Docker containers and distributed test execution.
Windows Batch Script
Execute native Windows CMD scripts. Perfect for legacy Windows automation, .NET builds, and Windows-specific toolchains.
Use Cases
@echo off set BUILD_CONFIG=Release msbuild MySolution.sln /p:Configuration=%BUILD_CONFIG% xcopy /s /y bin\Release dist\
Windows PowerShell
Leverage PowerShell's powerful scripting capabilities for advanced Windows automation with access to .NET libraries.
Use Cases
$ErrorActionPreference = "Stop"
Import-Module Az.Storage
Get-ChildItem -Recurse -Filter "*.dll" |
ForEach-Object { Sign-Code $_.FullName }Shell Script (Bash)
Cross-platform shell scripting for Linux and macOS agents. The workhorse of CI/CD pipelines worldwide.
Use Cases
#!/bin/bash set -e npm ci npm run build npm run test -- --coverage tar -czf dist.tar.gz ./dist
Python Script
Write build logic in Python with full access to pip packages. Ideal for data processing, API calls, and complex automation.
Use Cases
import subprocess
import json
# Run tests and collect results
result = subprocess.run(
["pytest", "-v", "--json-report"],
capture_output=True
)
print(f"Exit code: {result.returncode}")Docker Container
Run scripts inside a Docker container with automatic workspace mounting. Ensure consistent environments across agents.
Use Cases
# Image: python:3.11-slim # Registry: ghcr.io (with auth) pip install -r requirements.txt pytest tests/ -v --junitxml=results.xml
Test Provider (Pytest)
Native Pytest integration with parallel test distribution. Automatically split tests across multiple agents with label-based routing.
Use Cases
Provider: pytest Path: tests/ Args: -m "smoke and not slow" Max Agents: 4 Tests per batch: 10 Restrictions: tests/gpu/* → [gpu-agent] tests/browser/* → [browser, windows]
Copy Artifacts
Retrieve artifacts from another job's builds. Build dependencies between jobs or reuse compiled binaries.
Use Cases
Source Job: build-core-libs Which Build: Latest Successful Artifacts: libs/*.dll, configs/*.json Target: ./dependencies/
Docker Container Execution
First-class Docker support
Run your builds in isolated Docker containers with automatic workspace mounting. Supports private registries and authentication for enterprise environments.
- Any Docker Hub or private registry image
- Automatic workspace volume mounting
- Registry authentication with stored credentials
- Shell scripts run inside the container
- type: docker
image: python:3.11-slim
registry_url: ghcr.io
auth_user_id: docker-registry-creds
script: |
pip install -r requirements.txt
pytest tests/ -v --junitxml=results.xml
always_run: false9 Dedicated Configuration Tabs
Every aspect of your CI/CD pipeline is configurable through a dedicated tab in the job editor. From source control to post-build notifications — full control at your fingertips.
General
- Job name, description, project assignment
- Workspace path & cleanup options (git files, untracked)
- Build rotation: days to keep, max builds
- Jira integration: auto-create issues on failure
- 4 parameter types: String, Boolean, Choice, SCM
- Concurrent build execution toggle
Source Management
- Multiple SCM providers: Git, Bitbucket, GitLab
- Project/Organization and Repository selection
- Branch expressions with variable support
- Recursive submodule updates
- SSH key authentication override
- Custom SSH port configuration
AI Analysis
- Enable/disable AI-powered test analysis
- Process monitoring for crash detection
- Log file patterns for ingestion
- Screenshots and video collection rules
- Custom AI model configuration
- Triage automation settings
Triggers
- Cron-style scheduled builds
- Remote trigger URL with auth token
- Poll SCM for changes periodically
- Trigger after another job finishes
- Trigger even if upstream is unstable
- Multiple cron expressions supported
Restrictions
- Node label expressions for routing
- Restrict to specific agents/labels
- Wait for previous build to finish
- Matrix child build routing
- Concurrency control across jobs
- Build name pattern matching
Matrix Configuration
- Nodes axis: select agents by name/label
- Custom axis: define value lists
- Up to 2 axes for parallel execution
- Expandable tree for node selection
- Default values for single runs
- Combine axes for full permutations
Build Environment
- Custom build name with variables
- Python: VirtualEnv or PyEnv setup
- Environment storage: workspace or global
- pip requirements.txt support
- Custom environment variables injection
- Base Python interpreter selection
Build Scripts
- 7 step types: Batch, PowerShell, Bash, Python, Docker, Test Provider, Copy Artifacts
- Drag-and-drop step reordering
- "Always run" option for cleanup steps
- Docker with registry authentication
- Copy artifacts from other jobs
- Test provider with parallel execution
Post-build Actions
- Email notifications with templates
- HTTP/S webhooks (GET, POST, PUT, DELETE)
- Custom headers and body for webhooks
- Send from server or node
- Environment variable substitution
- Conditional: send only for main task
Understand Every Test Result
TestHide's report page goes beyond simple pass/fail. Every test is enriched with AI insights, visual evidence, resolution tracking, and historical context.
Resolution Workflow
Visual Evidence
- Screenshots with full-screen view
- Video recordings with playback
- Inline preview in test detail
- Download individual or all
Traceback & Logs
- Full stack traces with syntax highlighting
- Log aggregation from multiple files
- Search and filter within logs
- Console output capture
AI Triage Badges
- Root Cause classification badge
- Flakiness probability indicator
- OOD (novel failure) warning
- Impacted by recent changes
Test History
- Pass/fail timeline across builds
- Flakiness trend visualization
- Duration anomaly detection
- First failure identification
Attachments Hub
- All artifacts in sidebar
- Bulk download as ZIP
- File type detection
- Deep link to specific files
Technical Insights
- Code metrics (LOC, complexity)
- Assert density analysis
- Health score per file
- Code snippets with findings
Flakiness Categories
Automatic classification based on failure patterns
Powerful Agent Capabilities
TestHide's .NET agent is a full-featured execution engine with remote control, system monitoring, and intelligent task management — all accessible from the dashboard.
Remote Desktop
- Live screen streaming
- Mouse click/drag/scroll
- Keyboard input simulation
- Full remote control from browser
System Monitoring
- Free RAM & Disk space
- Client & Python version
- IP address & architecture
- Connection status tracking
Node Management
- Remote restart command
- Force client update
- Disk cleanup utility
- Fetch agent logs remotely
Label-Based Routing
- Custom labels per agent
- Job routing by labels
- Test restrictions (gpu, browser)
- Environment variables
Concurrent Execution
- Multiple jobs per agent
- Exclusive vs concurrent modes
- Task queue management
- Resource-aware scheduling
License Management
- Per-agent licensing
- License tier display
- Automatic validation
- Capacity limits per node
Node View Dashboard
Monitor and manage each agent from the UI
Status
System Details
testhide_client
Cross-platform .NET agent
A lightweight, self-updating agent written in C# that runs on Windows, Linux, and macOS. Handles build execution, artifact collection, and real-time communication via WebSocket.
- Auto-update from server on new versions
- Persistent startup (runs on boot)
- Local AI analysis after builds
- Graceful shutdown on OS signals
- Lock file management for task tracking
# Configure the agent URL on first run > testhide_client.exe config --url wss://testhide.example.com/ws ✓ Configuration saved to config.json WebSocket URL: wss://testhide.example.com/ws Instance ID: auto-generated # The agent will now auto-connect on startup > testhide_client.exe [INFO] Connected to TestHide server [INFO] Registered as: WIN-RUNNER-01 [INFO] Labels: [browser, windows] [INFO] Waiting for tasks...
Integrations
First-class integrations with Git providers, issue trackers, and notification systems. TestHide fits into your existing workflow.
- Git: Bitbucket, GitLab, GitHub webhooks
- Issue tracking: Jira ticket linking
- Notifications: MS Teams, Email, Slack
- Infrastructure: ESXi VM management
- API: Full REST API for custom integrations
Integrations Active: ✓ Bitbucket (webhooks enabled) ✓ Jira (auto-link failures) ✓ MS Teams (build notifications) ✓ Email (daily digest)
Ready to Experience These Features?
Get started with TestHide in minutes. Deploy with Docker or run locally.
Get Started