AYI-NEDJIMI Claude Sonnet 4.5 commited on
Commit Β·
14b051b
0
Parent(s):
Initial commit: Dataset Explorer v1.0
Browse files- Created comprehensive Gradio Space for exploring 80 cybersecurity datasets
- Features include search, filter, statistics, and visualizations
- Dark theme with Plotly charts
- Export to CSV/JSON
- Categories: AI, Defensive, Offensive, Compliance
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- .gitignore +30 -0
- README.md +208 -0
- app.py +508 -0
- requirements.txt +4 -0
.gitignore
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.py[cod]
|
| 3 |
+
*$py.class
|
| 4 |
+
*.so
|
| 5 |
+
.Python
|
| 6 |
+
build/
|
| 7 |
+
develop-eggs/
|
| 8 |
+
dist/
|
| 9 |
+
downloads/
|
| 10 |
+
eggs/
|
| 11 |
+
.eggs/
|
| 12 |
+
lib/
|
| 13 |
+
lib64/
|
| 14 |
+
parts/
|
| 15 |
+
sdist/
|
| 16 |
+
var/
|
| 17 |
+
wheels/
|
| 18 |
+
*.egg-info/
|
| 19 |
+
.installed.cfg
|
| 20 |
+
*.egg
|
| 21 |
+
.env
|
| 22 |
+
.venv
|
| 23 |
+
env/
|
| 24 |
+
venv/
|
| 25 |
+
ENV/
|
| 26 |
+
env.bak/
|
| 27 |
+
venv.bak/
|
| 28 |
+
.DS_Store
|
| 29 |
+
*.log
|
| 30 |
+
flagged/
|
README.md
ADDED
|
@@ -0,0 +1,208 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: Dataset Explorer
|
| 3 |
+
emoji: π
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: purple
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: 4.44.1
|
| 8 |
+
app_file: app.py
|
| 9 |
+
pinned: false
|
| 10 |
+
license: apache-2.0
|
| 11 |
+
tags:
|
| 12 |
+
- cybersecurity
|
| 13 |
+
- datasets
|
| 14 |
+
- data-explorer
|
| 15 |
+
- analytics
|
| 16 |
+
- visualization
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# π Cybersecurity Dataset Explorer
|
| 20 |
+
|
| 21 |
+
A comprehensive Gradio Space to explore and analyze 80+ cybersecurity datasets from HuggingFace.
|
| 22 |
+
|
| 23 |
+
## Features
|
| 24 |
+
|
| 25 |
+
### π Search & Filter
|
| 26 |
+
- Search by keyword across dataset names, descriptions, and tags
|
| 27 |
+
- Filter by language (English, Chinese, Korean, Italian, French, Russian, etc.)
|
| 28 |
+
- Filter by category (AI, Defensive, Offensive, Compliance)
|
| 29 |
+
- Filter by popularity (minimum downloads and likes)
|
| 30 |
+
- View results in interactive tables
|
| 31 |
+
|
| 32 |
+
### π Dataset Details
|
| 33 |
+
- Comprehensive metadata for each dataset
|
| 34 |
+
- Statistics (downloads, likes, size, language)
|
| 35 |
+
- Complete tag listings
|
| 36 |
+
- Direct links to HuggingFace repositories
|
| 37 |
+
- Mock preview functionality (shows structure)
|
| 38 |
+
|
| 39 |
+
### π Statistics & Visualizations
|
| 40 |
+
Interactive charts powered by Plotly:
|
| 41 |
+
- **Category Distribution**: Pie chart showing dataset distribution across categories
|
| 42 |
+
- **Language Distribution**: Bar chart of top 10 languages
|
| 43 |
+
- **Top Downloads**: Horizontal bar chart of most popular datasets
|
| 44 |
+
- **Size Distribution**: Distribution of dataset sizes
|
| 45 |
+
|
| 46 |
+
### π₯ Export Capabilities
|
| 47 |
+
- Export filtered results to CSV format
|
| 48 |
+
- Export filtered results to JSON format
|
| 49 |
+
- Download data for offline analysis
|
| 50 |
+
|
| 51 |
+
### π¨ Dark Theme
|
| 52 |
+
Beautiful dark theme optimized for readability with:
|
| 53 |
+
- High contrast colors
|
| 54 |
+
- Interactive hover effects
|
| 55 |
+
- Responsive layout
|
| 56 |
+
- Professional visualization styling
|
| 57 |
+
|
| 58 |
+
## Dataset Categories
|
| 59 |
+
|
| 60 |
+
### AI (27 datasets)
|
| 61 |
+
Datasets for training and evaluating AI/ML models in cybersecurity:
|
| 62 |
+
- Instruction-tuning datasets
|
| 63 |
+
- ShareGPT format conversations
|
| 64 |
+
- Question-answering pairs
|
| 65 |
+
- Synthetic training data
|
| 66 |
+
- Fine-tuning datasets
|
| 67 |
+
|
| 68 |
+
### Defensive (28 datasets)
|
| 69 |
+
Blue team, security operations, and threat detection:
|
| 70 |
+
- Threat intelligence
|
| 71 |
+
- Incident response
|
| 72 |
+
- Security operations
|
| 73 |
+
- Detection rules (SIGMA, YARA, Suricata)
|
| 74 |
+
- Honeypot data
|
| 75 |
+
- News and threat feeds
|
| 76 |
+
|
| 77 |
+
### Offensive (10 datasets)
|
| 78 |
+
Red team, penetration testing, and security research:
|
| 79 |
+
- Penetration testing techniques
|
| 80 |
+
- Exploit databases
|
| 81 |
+
- Attack scenarios
|
| 82 |
+
- Vulnerability data
|
| 83 |
+
- CVE databases
|
| 84 |
+
|
| 85 |
+
### Compliance (5 datasets)
|
| 86 |
+
Regulatory frameworks and standards:
|
| 87 |
+
- NIST Cybersecurity Framework
|
| 88 |
+
- ISO/IEC 27001
|
| 89 |
+
- Taiwan Cybersecurity Law
|
| 90 |
+
- Compliance training data
|
| 91 |
+
|
| 92 |
+
## Top Datasets
|
| 93 |
+
|
| 94 |
+
1. **ethanolivertroy/nist-cybersecurity-training** (8,000 downloads)
|
| 95 |
+
- Largest open-source NIST cybersecurity training dataset
|
| 96 |
+
- 100K-1M samples for LLM fine-tuning
|
| 97 |
+
|
| 98 |
+
2. **clydeiii/cybersecurity** (4,000 downloads)
|
| 99 |
+
- APT notes from GitHub
|
| 100 |
+
- Threat intelligence focus
|
| 101 |
+
|
| 102 |
+
3. **vinitvek/cybersecurityattacks** (2,300 downloads)
|
| 103 |
+
- Cybersecurity attacks dataset
|
| 104 |
+
- 10K-100K samples
|
| 105 |
+
|
| 106 |
+
4. **Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset** (786 downloads, 78 likes)
|
| 107 |
+
- 53,202 instruction-tuning examples
|
| 108 |
+
- Defensive security focus
|
| 109 |
+
|
| 110 |
+
5. **AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.0** (353 downloads)
|
| 111 |
+
- 83,920 high-quality training triples
|
| 112 |
+
- Defensive cybersecurity
|
| 113 |
+
|
| 114 |
+
## Statistics
|
| 115 |
+
|
| 116 |
+
- **Total Datasets**: 80
|
| 117 |
+
- **Total Downloads**: 18,000+
|
| 118 |
+
- **Languages**: 10+ (English, Chinese, Korean, Italian, French, Russian, etc.)
|
| 119 |
+
- **Size Range**: <1K to 10M+ samples
|
| 120 |
+
|
| 121 |
+
## Usage
|
| 122 |
+
|
| 123 |
+
### Search Examples
|
| 124 |
+
|
| 125 |
+
1. **Find NIST-related datasets**:
|
| 126 |
+
- Keyword: "NIST"
|
| 127 |
+
- Category: Compliance
|
| 128 |
+
|
| 129 |
+
2. **Find penetration testing datasets**:
|
| 130 |
+
- Keyword: "penetration" or "pentest"
|
| 131 |
+
- Category: Offensive
|
| 132 |
+
|
| 133 |
+
3. **Find instruction-tuning datasets**:
|
| 134 |
+
- Keyword: "instruction"
|
| 135 |
+
- Category: AI
|
| 136 |
+
- Min Downloads: 100
|
| 137 |
+
|
| 138 |
+
4. **Find threat intelligence datasets**:
|
| 139 |
+
- Keyword: "threat"
|
| 140 |
+
- Category: Defensive
|
| 141 |
+
|
| 142 |
+
### Export Workflow
|
| 143 |
+
|
| 144 |
+
1. Apply desired filters
|
| 145 |
+
2. Click "Search Datasets"
|
| 146 |
+
3. Click "Export to CSV" or "Export to JSON"
|
| 147 |
+
4. Download the file from the interface
|
| 148 |
+
|
| 149 |
+
## Technologies
|
| 150 |
+
|
| 151 |
+
- **Gradio 4.44.1**: Interactive web interface
|
| 152 |
+
- **Pandas 2.1.4**: Data manipulation and filtering
|
| 153 |
+
- **Plotly 5.18.0**: Interactive visualizations
|
| 154 |
+
- **HuggingFace Datasets 2.16.1**: Dataset metadata
|
| 155 |
+
|
| 156 |
+
## Data Sources
|
| 157 |
+
|
| 158 |
+
All datasets are publicly available on HuggingFace Hub. This explorer provides:
|
| 159 |
+
- Curated metadata from 80 cybersecurity datasets
|
| 160 |
+
- Filtering and search capabilities
|
| 161 |
+
- Visual analytics
|
| 162 |
+
- Export functionality
|
| 163 |
+
|
| 164 |
+
To access actual dataset content, click the HuggingFace URL for any dataset.
|
| 165 |
+
|
| 166 |
+
## Development
|
| 167 |
+
|
| 168 |
+
### Local Setup
|
| 169 |
+
|
| 170 |
+
```bash
|
| 171 |
+
pip install -r requirements.txt
|
| 172 |
+
python app.py
|
| 173 |
+
```
|
| 174 |
+
|
| 175 |
+
### File Structure
|
| 176 |
+
|
| 177 |
+
```
|
| 178 |
+
dataset-explorer/
|
| 179 |
+
βββ app.py # Main Gradio application
|
| 180 |
+
βββ requirements.txt # Python dependencies
|
| 181 |
+
βββ README.md # This file
|
| 182 |
+
```
|
| 183 |
+
|
| 184 |
+
## Future Enhancements
|
| 185 |
+
|
| 186 |
+
- Live dataset preview (load actual samples)
|
| 187 |
+
- Full-text search within dataset content
|
| 188 |
+
- Advanced filtering (by date, size range)
|
| 189 |
+
- Dataset comparison tool
|
| 190 |
+
- API integration for real-time updates
|
| 191 |
+
- Custom visualization builder
|
| 192 |
+
- Dataset recommendation engine
|
| 193 |
+
|
| 194 |
+
## License
|
| 195 |
+
|
| 196 |
+
Apache 2.0
|
| 197 |
+
|
| 198 |
+
## Author
|
| 199 |
+
|
| 200 |
+
**AYI-NEDJIMI**
|
| 201 |
+
|
| 202 |
+
## Acknowledgments
|
| 203 |
+
|
| 204 |
+
Special thanks to the HuggingFace community and all dataset creators who make their cybersecurity datasets publicly available.
|
| 205 |
+
|
| 206 |
+
---
|
| 207 |
+
|
| 208 |
+
**Note**: This is a metadata explorer. To download and use the actual datasets, visit the HuggingFace links provided in the interface.
|
app.py
ADDED
|
@@ -0,0 +1,508 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import gradio as gr
|
| 2 |
+
import pandas as pd
|
| 3 |
+
import plotly.express as px
|
| 4 |
+
import plotly.graph_objects as go
|
| 5 |
+
from datasets import load_dataset
|
| 6 |
+
import json
|
| 7 |
+
from datetime import datetime
|
| 8 |
+
import re
|
| 9 |
+
|
| 10 |
+
# Dataset metadata from HuggingFace search
|
| 11 |
+
DATASETS_METADATA = [
|
| 12 |
+
{"name": "ethanolivertroy/nist-cybersecurity-training", "downloads": 8000, "likes": 48, "size": "100K-1M", "language": "en", "tags": ["cybersecurity", "nist", "compliance", "security-controls", "zero-trust", "privacy"], "category": "compliance", "description": "NIST Cybersecurity Training Dataset v1.1 - The largest open-source NIST cybersecurity training dataset for fine-tuning LLMs"},
|
| 13 |
+
{"name": "clydeiii/cybersecurity", "downloads": 4000, "likes": 6, "size": "100K-1M", "language": "unknown", "tags": ["APT", "threat-intelligence"], "category": "offensive", "description": "APT notes dataset from GitHub"},
|
| 14 |
+
{"name": "vinitvek/cybersecurityattacks", "downloads": 2300, "likes": 5, "size": "10K-100K", "language": "en", "tags": ["attacks", "security"], "category": "offensive", "description": "Cybersecurity attacks dataset"},
|
| 15 |
+
{"name": "Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset", "downloads": 786, "likes": 78, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "defensive-security", "instruction-tuning", "threat-intelligence", "incident-response", "security-operations"], "category": "defensive", "description": "53,202 meticulously curated system/user/assistant instruction-tuning examples covering defensive security"},
|
| 16 |
+
{"name": "AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.0", "downloads": 353, "likes": 10, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "defensive-security", "instruction-tuning"], "category": "defensive", "description": "83,920 high-quality system/user/assistant triples for defensive cybersecurity"},
|
| 17 |
+
{"name": "AlicanKiraz0/Cybersecurity-Dataset-Heimdall-v1.1", "downloads": 192, "likes": 13, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "defensive-security", "instruction-tuning"], "category": "defensive", "description": "21,258 high-quality system/user/assistant triples for training alignment-safe, defensive-cybersecurity LLMs"},
|
| 18 |
+
{"name": "Chemically-motivated/CyberSecurityDataset", "downloads": 180, "likes": 3, "size": "<1K", "language": "en", "tags": ["cybersecurity", "machine learning", "pentesting", "exploits"], "category": "offensive", "description": "Curated data points related to penetration testing, known exploits, and vulnerabilities"},
|
| 19 |
+
{"name": "ChaoticNeutrals/Cybersecurity-ShareGPT", "downloads": 175, "likes": 15, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "ShareGPT"], "category": "ai", "description": "Converted, deslopped, min-hash deduplicated cybersecurity ShareGPT dataset"},
|
| 20 |
+
{"name": "Mohabahmed03/Alpaca_Dataset_CyberSecurity_Smaller_2.0", "downloads": 145, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["Cyber", "Security", "Cybersecurity", "LLM", "Pentest", "RedTeam", "BlueTeam"], "category": "ai", "description": "Alpaca format cybersecurity dataset"},
|
| 21 |
+
{"name": "Bouquets/DeepSeek-V3-Distill-Cybersecurity-en", "downloads": 140, "likes": 0, "size": "1K-10K", "language": "en", "tags": ["cybersecurity", "penetration-testing", "distilled"], "category": "offensive", "description": "High-quality distilled dataset specialized in cybersecurity penetration testing domain"},
|
| 22 |
+
{"name": "Druva-S-Kumar/cybersecurity-qa-dataset", "downloads": 123, "likes": 0, "size": "<1K", "language": "en", "tags": ["qa", "cybersecurity"], "category": "ai", "description": "Cybersecurity Q&A dataset"},
|
| 23 |
+
{"name": "Rowden/CybersecurityQAA", "downloads": 119, "likes": 4, "size": "1K-10K", "language": "en", "tags": ["cybersecurity", "QAA"], "category": "ai", "description": "Cybersecurity Question-Answer-Assertion (QAA) Dataset designed to evaluate LLM capabilities"},
|
| 24 |
+
{"name": "luckwa/cybersecurity-dataset", "downloads": 119, "likes": 1, "size": "1K-10K", "language": "en", "tags": ["cybersecurity"], "category": "defensive", "description": "General cybersecurity dataset"},
|
| 25 |
+
{"name": "Vanessasml/cybersecurity_32k_instruction_input_output", "downloads": 114, "likes": 17, "size": "10K-100K", "language": "en", "tags": ["NIST", "ITC EBA", "threat-identification"], "category": "compliance", "description": "Q&As focused on identification of cyber threats, and text classification under NIST taxonomy"},
|
| 26 |
+
{"name": "AlicanKiraz0/Cybersecurity-Dataset-v1", "downloads": 98, "likes": 12, "size": "1K-10K", "language": "en", "tags": ["cybersecurity"], "category": "defensive", "description": "2,500 high-quality instruction-response pairs focused on defensive cybersecurity education"},
|
| 27 |
+
{"name": "mariiazhiv/cybersecurity_qa", "downloads": 97, "likes": 1, "size": "<1K", "language": "en", "tags": ["question-answering", "cybersecurity"], "category": "ai", "description": "Instruction-response pairs focused on cybersecurity concepts"},
|
| 28 |
+
{"name": "CyberNative/CyberSecurityEval", "downloads": 84, "likes": 19, "size": "<1K", "language": "en", "tags": ["cybersecurity", "infosec", "IT", "evaluation"], "category": "ai", "description": "CyberNative AI for CyberSecurity Q/A Evaluation - NOT FOR TRAINING"},
|
| 29 |
+
{"name": "whybe-choi/kovidore-v2-cybersecurity-beir", "downloads": 80, "likes": 1, "size": "1K-10K", "language": "ko", "tags": ["Visual Retrieving", "Industrial RAG"], "category": "defensive", "description": "Corpus of technical reports on cyber threat trends and security incident responses in Korea"},
|
| 30 |
+
{"name": "Canstralian/Purple-Team-Cybersecurity-Dataset", "downloads": 73, "likes": 9, "size": "10K-100K", "language": "en", "tags": ["purple-team", "code"], "category": "defensive", "description": "Synthetic collection designed to simulate collaborative cybersecurity exercises"},
|
| 31 |
+
{"name": "Bouquets/Cybersecurity-LLM-CVE", "downloads": 46, "likes": 15, "size": "100K-1M", "language": "en", "tags": ["CVE", "vulnerabilities"], "category": "defensive", "description": "CVE vulnerability database for cybersecurity"},
|
| 32 |
+
{"name": "theResearchNinja/benchmarkResults_violentUTF_cybersecurityBehavior", "downloads": 37, "likes": 1, "size": "100K-1M", "language": "en", "tags": ["benchmark", "results"], "category": "ai", "description": "Interdependent cybersecurity benchmark results"},
|
| 33 |
+
{"name": "schooly/Cyber-Security-Breaches", "downloads": 36, "likes": 11, "size": "1K-10K", "language": "en", "tags": ["breaches", "incidents"], "category": "offensive", "description": "Cyber security breaches dataset"},
|
| 34 |
+
{"name": "jcordon5/cybersecurity-rules", "downloads": 36, "likes": 9, "size": "<1K", "language": "en", "tags": ["SIGMA", "YARA", "Suricata", "detection-rules"], "category": "defensive", "description": "950 detection rules from official SIGMA, YARA, and Suricata repositories"},
|
| 35 |
+
{"name": "Tiamz/cybersecurity-instruction-dataset", "downloads": 33, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["instruction", "cybersecurity"], "category": "ai", "description": "Cybersecurity instruction dataset"},
|
| 36 |
+
{"name": "zeroshot/cybersecurity-corpus", "downloads": 29, "likes": 9, "size": "1K-10K", "language": "en", "tags": ["corpus"], "category": "ai", "description": "Cybersecurity corpus for training"},
|
| 37 |
+
{"name": "mteb/kovidore-v2-cybersecurity-mteb", "downloads": 29, "likes": 0, "size": "1K-10K", "language": "ko", "tags": ["MTEB", "retrieval"], "category": "ai", "description": "MTEB cybersecurity retrieval dataset in Korean"},
|
| 38 |
+
{"name": "electricsheepafrica/nigerian-telecom-cybersecurity-incident-logs", "downloads": 27, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["telecom", "cybersecurity", "incident", "logs"], "category": "defensive", "description": "Security events including intrusions, DDoS attacks, and malware on telecom infrastructure"},
|
| 39 |
+
{"name": "CyberNative/github_cybersecurity_READMEs", "downloads": 26, "likes": 14, "size": "1K-10K", "language": "en", "tags": ["github", "README"], "category": "ai", "description": "GitHub cybersecurity README files"},
|
| 40 |
+
{"name": "Mohabahmed03/Alpaca_Dataset_CyberSecurity_2.0", "downloads": 26, "likes": 0, "size": "100K-1M", "language": "en", "tags": ["Cyber", "Security", "Pentest", "Cybersecurity", "LLM", "BlueTeam"], "category": "ai", "description": "Alpaca format cybersecurity dataset v2.0"},
|
| 41 |
+
{"name": "hcnote/Cybersecurity-Dataset", "downloads": 26, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["code", "question-answering"], "category": "ai", "description": "High-quality cybersecurity dataset"},
|
| 42 |
+
{"name": "Zeo6/CyberSecurity-FineTune", "downloads": 25, "likes": 0, "size": "unknown", "language": "en", "tags": ["finetune"], "category": "ai", "description": "Cybersecurity fine-tuning dataset"},
|
| 43 |
+
{"name": "ystemsrx/Cybersecurity-ShareGPT-Chinese", "downloads": 24, "likes": 21, "size": "10K-100K", "language": "zh", "tags": ["code", "Chinese"], "category": "ai", "description": "Chinese cybersecurity dataset in ShareGPT format"},
|
| 44 |
+
{"name": "whybe-choi/kovidore-v2-cybersecurity-mteb", "downloads": 24, "likes": 0, "size": "1K-10K", "language": "ko", "tags": ["MTEB", "retrieval"], "category": "ai", "description": "MTEB cybersecurity retrieval dataset"},
|
| 45 |
+
{"name": "princemaxp/cybersecurity-keywords", "downloads": 22, "likes": 1, "size": "<1K", "language": "en", "tags": ["cybersecurity", "keywords"], "category": "ai", "description": "Common cybersecurity keywords list"},
|
| 46 |
+
{"name": "madox81/cybersecurity_attack_conversational_dataset", "downloads": 20, "likes": 0, "size": "unknown", "language": "en", "tags": ["conversational", "attacks"], "category": "offensive", "description": "Conversational cybersecurity attack dataset"},
|
| 47 |
+
{"name": "safouene99999/Cybersecurity_QA", "downloads": 19, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["QA"], "category": "ai", "description": "Cybersecurity Q&A dataset"},
|
| 48 |
+
{"name": "hcnote/High-quality-cybersecurity-datasets", "downloads": 19, "likes": 0, "size": "100K-1M", "language": "en", "tags": ["high-quality"], "category": "ai", "description": "277,707 high-quality cybersecurity records with AI annotation"},
|
| 49 |
+
{"name": "theResearchNinja/violentutf_cybersecurityBehavior", "downloads": 18, "likes": 3, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "cognitive behavioral psychology", "benchmark"], "category": "ai", "description": "LLM cybersecurity behavior benchmark dataset"},
|
| 50 |
+
{"name": "GotThatData/nist-cybersecurity-framework", "downloads": 18, "likes": 7, "size": "1K-10K", "language": "en", "tags": ["NIST", "Cybersecurity", "Framework"], "category": "compliance", "description": "NIST Cybersecurity Publications Dataset"},
|
| 51 |
+
{"name": "Mohabahmed03/Alpaca_Dataset_General_CyberSecurity", "downloads": 18, "likes": 0, "size": "100K-1M", "language": "en", "tags": ["General", "Alpaca", "CyberSecurity"], "category": "ai", "description": "General Alpaca format cybersecurity dataset"},
|
| 52 |
+
{"name": "vnovaai19/CYBERSECURITY_JSONL_V1", "downloads": 18, "likes": 0, "size": "<1K", "language": "en", "tags": ["cybersecurity", "synthetic-data", "safety", "phishing", "fraud-detection"], "category": "defensive", "description": "100 synthetic cybersecurity threat scenarios with educational AI responses"},
|
| 53 |
+
{"name": "Mattimax/Cybersecurity-ShareGPT-Italian", "downloads": 18, "likes": 0, "size": "1K-10K", "language": "it", "tags": ["Italian", "ShareGPT"], "category": "ai", "description": "Italian cybersecurity ShareGPT dataset"},
|
| 54 |
+
{"name": "olgazigbeehub/cybersecurity-news-dataset-english-3000", "downloads": 18, "likes": 0, "size": "1K-10K", "language": "en", "tags": ["news", "cybersecurity", "media-analysis"], "category": "defensive", "description": "3,000 English-language cybersecurity news metadata rows"},
|
| 55 |
+
{"name": "hcnote/Cybersecurity-High-Quality-Dataset", "downloads": 17, "likes": 0, "size": "100K-1M", "language": "en", "tags": ["high-quality"], "category": "ai", "description": "270,271 high-quality Chinese-English Q&A cybersecurity dataset"},
|
| 56 |
+
{"name": "ScoutieAutoML/cybersecurity_news_telegram_dataset", "downloads": 16, "likes": 2, "size": "10K-100K", "language": "ru", "tags": ["russia", "cybersecurity", "media", "news"], "category": "defensive", "description": "Russian-language Telegram news channels on cybersecurity"},
|
| 57 |
+
{"name": "savaniDhruv/Cybersecurity_Attack_Dataset", "downloads": 16, "likes": 2, "size": "10K-100K", "language": "en", "tags": ["attacks"], "category": "offensive", "description": "Cybersecurity attack dataset"},
|
| 58 |
+
{"name": "pyToshka/cyber-security-events", "downloads": 16, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "honeypot", "threat-intelligence"], "category": "defensive", "description": "Cybersecurity events collected from honeypot infrastructure"},
|
| 59 |
+
{"name": "ahmadkaab/Trendyol-Cybersecurity-Instruction-Tuning-Dataset", "downloads": 16, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "defensive-security", "instruction-tuning"], "category": "defensive", "description": "53,202 defensive security instruction-tuning examples"},
|
| 60 |
+
{"name": "MCP-1st-Birthday/smoltrace-cybersecurity-tasks", "downloads": 15, "likes": 0, "size": "<1K", "language": "en", "tags": ["smoltrace", "synthetic-data", "agent-evaluation"], "category": "ai", "description": "SMOLTRACE synthetic dataset for agent evaluation"},
|
| 61 |
+
{"name": "ErebusTN/The-Ultimate-CyberSecurity-Dataset-Collection", "downloads": 14, "likes": 1, "size": "unknown", "language": "en", "tags": ["collection"], "category": "ai", "description": "Ultimate cybersecurity dataset collection"},
|
| 62 |
+
{"name": "NewsDataHub/cybersecurity-news-dataset-english-3000", "downloads": 14, "likes": 1, "size": "1K-10K", "language": "en", "tags": ["news", "cybersecurity"], "category": "defensive", "description": "3,000 English cybersecurity news metadata rows"},
|
| 63 |
+
{"name": "AYI-NEDJIMI/ai-cybersecurity-en", "downloads": 14, "likes": 0, "size": "<1K", "language": "en", "tags": ["artificial-intelligence", "cybersecurity", "offensive-ai", "defensive-ai", "deepfake"], "category": "ai", "description": "AI in Offensive and Defensive Cybersecurity - English Dataset"},
|
| 64 |
+
{"name": "AR2021/cybersecurity-corpus-llama2-1k", "downloads": 13, "likes": 1, "size": "<1K", "language": "en", "tags": ["llama2"], "category": "ai", "description": "Cybersecurity corpus for Llama2"},
|
| 65 |
+
{"name": "boapro/Purple-Team-Cybersecurity-Dataset", "downloads": 13, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["code", "purple-team"], "category": "defensive", "description": "Synthetic purple team cybersecurity exercises"},
|
| 66 |
+
{"name": "tuandunghcmut/Trendyol-Cybersecurity-Instruction-Tuning-Dataset", "downloads": 13, "likes": 1, "size": "10K-100K", "language": "en", "tags": ["cybersecurity", "security", "cyber-defense", "conversational"], "category": "defensive", "description": "GPT format conversational cybersecurity dataset"},
|
| 67 |
+
{"name": "AYI-NEDJIMI/ai-cybersecurity-fr", "downloads": 13, "likes": 0, "size": "<1K", "language": "fr", "tags": ["artificial-intelligence", "cybersecurity", "offensive-ai", "defensive-ai"], "category": "ai", "description": "AI in Offensive and Defensive Cybersecurity - French Dataset"},
|
| 68 |
+
{"name": "pki/autonlp-data-cybersecurity", "downloads": 12, "likes": 0, "size": "unknown", "language": "en", "tags": ["autonlp"], "category": "ai", "description": "AutoNLP cybersecurity data"},
|
| 69 |
+
{"name": "Hadihilman/cybersecurity-dataset", "downloads": 12, "likes": 0, "size": "<1K", "language": "en", "tags": ["images"], "category": "defensive", "description": "Cybersecurity image dataset"},
|
| 70 |
+
{"name": "AnodeAI/Elite_quality_cybersecurity", "downloads": 12, "likes": 1, "size": "10K-100K", "language": "en", "tags": ["legal", "finance"], "category": "ai", "description": "Elite quality cybersecurity dataset"},
|
| 71 |
+
{"name": "hcnote/Cybersecurity-bigDataset", "downloads": 12, "likes": 0, "size": "100K-1M", "language": "en", "tags": ["large-scale"], "category": "ai", "description": "Global first open-source mega-scale cybersecurity dataset"},
|
| 72 |
+
{"name": "bnsapa/cybersecurity-ner", "downloads": 11, "likes": 2, "size": "1K-10K", "language": "en", "tags": ["token-classification", "NER"], "category": "ai", "description": "Cybersecurity named entity recognition dataset"},
|
| 73 |
+
{"name": "baig31/Cybersecurity_penetration_testing_books", "downloads": 11, "likes": 12, "size": "unknown", "language": "en", "tags": ["books", "penetration-testing"], "category": "offensive", "description": "Cybersecurity penetration testing books"},
|
| 74 |
+
{"name": "beldua/english-cybersecurity-basics-30", "downloads": 11, "likes": 0, "size": "<1K", "language": "en", "tags": ["basics"], "category": "ai", "description": "English cybersecurity basics"},
|
| 75 |
+
{"name": "ahmedds10/finetuning_cybersecurity", "downloads": 10, "likes": 0, "size": "<1K", "language": "en", "tags": ["finetuning"], "category": "ai", "description": "Cybersecurity fine-tuning dataset"},
|
| 76 |
+
{"name": "Mohabahmed03/Alpaca_Dataset_CyberSecurity_Smaller", "downloads": 10, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["CyberSecurity", "Finetune"], "category": "ai", "description": "Smaller Alpaca cybersecurity dataset"},
|
| 77 |
+
{"name": "ChavyvAkvar/Trendyol-Cybersecurity-Instruction-Tuning-Dataset-Converted", "downloads": 10, "likes": 1, "size": "10K-100K", "language": "en", "tags": ["converted"], "category": "defensive", "description": "Converted Trendyol cybersecurity dataset"},
|
| 78 |
+
{"name": "tandevllc/cybersecurity-atom-rss-feeds-2025", "downloads": 10, "likes": 1, "size": "unknown", "language": "en", "tags": ["news", "rss", "feeds"], "category": "defensive", "description": "Cybersecurity Atom/RSS feeds 2025"},
|
| 79 |
+
{"name": "tandevllc/cybersecurity-wiki-slices", "downloads": 10, "likes": 1, "size": "10K-100K", "language": "en", "tags": ["wikipedia", "cybersecurity"], "category": "ai", "description": "Curated collection of English Wikipedia pages covering cybersecurity"},
|
| 80 |
+
{"name": "antitheft159/CybersecurityAttacks", "downloads": 9, "likes": 1, "size": "unknown", "language": "en", "tags": ["attacks"], "category": "offensive", "description": "Cybersecurity attacks dataset"},
|
| 81 |
+
{"name": "Tiamz/cybersecurity-raw-json-datasets", "downloads": 9, "likes": 0, "size": "unknown", "language": "en", "tags": ["raw", "json"], "category": "ai", "description": "Raw JSON cybersecurity datasets"},
|
| 82 |
+
{"name": "burpsuite/Cybersecurity-Dataset-v1", "downloads": 9, "likes": 0, "size": "1K-10K", "language": "en", "tags": ["cybersecurity"], "category": "defensive", "description": "2,500 defensive cybersecurity instruction-response pairs"},
|
| 83 |
+
{"name": "Deshaune/Global-Cybersecurity-Threats-2015_2024", "downloads": 8, "likes": 1, "size": "1K-10K", "language": "en", "tags": ["global-threats"], "category": "defensive", "description": "Global cybersecurity threats from 2015-2024"},
|
| 84 |
+
{"name": "oceancharcoal/Cybersecurity_attack_dataset", "downloads": 8, "likes": 0, "size": "10K-100K", "language": "en", "tags": ["attacks"], "category": "offensive", "description": "Cybersecurity attack dataset"},
|
| 85 |
+
{"name": "pyToshka/cyber-security-events-full", "downloads": 8, "likes": 0, "size": "100K-1M", "language": "en", "tags": ["cybersecurity", "honeypot", "threat-intelligence"], "category": "defensive", "description": "Full cybersecurity events from honeypot infrastructure"},
|
| 86 |
+
{"name": "dattaraj/rag_eval_cybersecurity", "downloads": 7, "likes": 0, "size": "<1K", "language": "en", "tags": ["RAG", "evaluation"], "category": "ai", "description": "RAG evaluation for cybersecurity"},
|
| 87 |
+
{"name": "lianghsun/tw-cybersecurity", "downloads": 7, "likes": 0, "size": "1K-10K", "language": "zh", "tags": ["Taiwan", "cybersecurity", "ISO-27001"], "category": "compliance", "description": "Taiwan cybersecurity dataset with ISO/IEC 27001"},
|
| 88 |
+
{"name": "mariiazhiv/Cybersecurity_messages", "downloads": 7, "likes": 0, "size": "1K-10K", "language": "en", "tags": ["messages"], "category": "ai", "description": "Cybersecurity messages dataset"},
|
| 89 |
+
{"name": "MichaelPrimez/cybersecurity-questionaire", "downloads": 6, "likes": 0, "size": "<1K", "language": "en", "tags": ["questionnaire", "synthetic", "distilabel"], "category": "ai", "description": "Cybersecurity questionnaire dataset"},
|
| 90 |
+
{"name": "lianghsun/tw-cybersecurity-chat", "downloads": 5, "likes": 0, "size": "1K-10K", "language": "zh", "tags": ["Taiwan", "cybersecurity", "chat"], "category": "ai", "description": "Taiwan cybersecurity chat dataset"},
|
| 91 |
+
{"name": "WhoIsShe/CyberSecurity-big", "downloads": 5, "likes": 1, "size": "1M-10M", "language": "en", "tags": ["large-scale"], "category": "ai", "description": "Large-scale cybersecurity dataset"},
|
| 92 |
+
]
|
| 93 |
+
|
| 94 |
+
def create_dataframe():
|
| 95 |
+
"""Create pandas DataFrame from metadata"""
|
| 96 |
+
df = pd.DataFrame(DATASETS_METADATA)
|
| 97 |
+
df['url'] = df['name'].apply(lambda x: f"https://huggingface.co/datasets/{x}")
|
| 98 |
+
return df
|
| 99 |
+
|
| 100 |
+
def get_dataset_stats():
|
| 101 |
+
"""Generate overall statistics"""
|
| 102 |
+
df = create_dataframe()
|
| 103 |
+
|
| 104 |
+
stats = {
|
| 105 |
+
"Total Datasets": len(df),
|
| 106 |
+
"Total Downloads": f"{df['downloads'].sum():,}",
|
| 107 |
+
"Total Likes": f"{df['likes'].sum():,}",
|
| 108 |
+
"Languages": len(df['language'].unique()),
|
| 109 |
+
"Categories": len(df['category'].unique()),
|
| 110 |
+
}
|
| 111 |
+
return stats
|
| 112 |
+
|
| 113 |
+
def filter_datasets(keyword, language, category, min_downloads, min_likes):
|
| 114 |
+
"""Filter datasets based on criteria"""
|
| 115 |
+
df = create_dataframe()
|
| 116 |
+
|
| 117 |
+
# Filter by keyword
|
| 118 |
+
if keyword:
|
| 119 |
+
mask = (
|
| 120 |
+
df['name'].str.contains(keyword, case=False, na=False) |
|
| 121 |
+
df['description'].str.contains(keyword, case=False, na=False) |
|
| 122 |
+
df['tags'].apply(lambda x: any(keyword.lower() in tag.lower() for tag in x))
|
| 123 |
+
)
|
| 124 |
+
df = df[mask]
|
| 125 |
+
|
| 126 |
+
# Filter by language
|
| 127 |
+
if language and language != "All":
|
| 128 |
+
df = df[df['language'] == language]
|
| 129 |
+
|
| 130 |
+
# Filter by category
|
| 131 |
+
if category and category != "All":
|
| 132 |
+
df = df[df['category'] == category]
|
| 133 |
+
|
| 134 |
+
# Filter by downloads
|
| 135 |
+
if min_downloads:
|
| 136 |
+
df = df[df['downloads'] >= min_downloads]
|
| 137 |
+
|
| 138 |
+
# Filter by likes
|
| 139 |
+
if min_likes:
|
| 140 |
+
df = df[df['likes'] >= min_likes]
|
| 141 |
+
|
| 142 |
+
return df
|
| 143 |
+
|
| 144 |
+
def search_datasets(keyword, language, category, min_downloads, min_likes):
|
| 145 |
+
"""Search and display datasets"""
|
| 146 |
+
df = filter_datasets(keyword, language, category, min_downloads, min_likes)
|
| 147 |
+
|
| 148 |
+
# Format for display
|
| 149 |
+
display_df = df[['name', 'downloads', 'likes', 'size', 'language', 'category', 'description']].copy()
|
| 150 |
+
display_df.columns = ['Dataset Name', 'Downloads', 'Likes', 'Size', 'Language', 'Category', 'Description']
|
| 151 |
+
|
| 152 |
+
result_text = f"Found {len(df)} datasets matching your criteria"
|
| 153 |
+
|
| 154 |
+
return display_df, result_text
|
| 155 |
+
|
| 156 |
+
def get_dataset_details(dataset_name):
|
| 157 |
+
"""Get detailed information about a specific dataset"""
|
| 158 |
+
df = create_dataframe()
|
| 159 |
+
|
| 160 |
+
if not dataset_name:
|
| 161 |
+
return "Please select a dataset from the list above", None, None
|
| 162 |
+
|
| 163 |
+
dataset = df[df['name'] == dataset_name]
|
| 164 |
+
|
| 165 |
+
if dataset.empty:
|
| 166 |
+
return "Dataset not found", None, None
|
| 167 |
+
|
| 168 |
+
dataset = dataset.iloc[0]
|
| 169 |
+
|
| 170 |
+
details = f"""
|
| 171 |
+
## {dataset['name']}
|
| 172 |
+
|
| 173 |
+
**Description:** {dataset['description']}
|
| 174 |
+
|
| 175 |
+
**Statistics:**
|
| 176 |
+
- Downloads: {dataset['downloads']:,}
|
| 177 |
+
- Likes: {dataset['likes']}
|
| 178 |
+
- Size: {dataset['size']}
|
| 179 |
+
- Language: {dataset['language']}
|
| 180 |
+
- Category: {dataset['category']}
|
| 181 |
+
|
| 182 |
+
**Tags:** {', '.join(dataset['tags'])}
|
| 183 |
+
|
| 184 |
+
**HuggingFace URL:** [{dataset['url']}]({dataset['url']})
|
| 185 |
+
|
| 186 |
+
---
|
| 187 |
+
|
| 188 |
+
*Note: To preview dataset samples, you would need to load the actual dataset using the HuggingFace datasets library.
|
| 189 |
+
This demo shows metadata only. For full dataset access, click the URL above.*
|
| 190 |
+
"""
|
| 191 |
+
|
| 192 |
+
# Create a simple preview table (mock data since we're not loading actual datasets)
|
| 193 |
+
preview_data = {
|
| 194 |
+
"Column": ["Feature 1", "Feature 2", "Feature 3"],
|
| 195 |
+
"Type": ["text", "text", "category"],
|
| 196 |
+
"Sample": ["Sample data...", "Sample data...", "Sample category..."]
|
| 197 |
+
}
|
| 198 |
+
preview_df = pd.DataFrame(preview_data)
|
| 199 |
+
|
| 200 |
+
return details, preview_df, dataset['url']
|
| 201 |
+
|
| 202 |
+
def create_category_chart():
|
| 203 |
+
"""Create pie chart of datasets by category"""
|
| 204 |
+
df = create_dataframe()
|
| 205 |
+
category_counts = df['category'].value_counts()
|
| 206 |
+
|
| 207 |
+
fig = px.pie(
|
| 208 |
+
values=category_counts.values,
|
| 209 |
+
names=category_counts.index,
|
| 210 |
+
title='Datasets by Category',
|
| 211 |
+
color_discrete_sequence=px.colors.sequential.RdBu,
|
| 212 |
+
hole=0.3
|
| 213 |
+
)
|
| 214 |
+
fig.update_layout(
|
| 215 |
+
paper_bgcolor='rgba(0,0,0,0)',
|
| 216 |
+
plot_bgcolor='rgba(0,0,0,0)',
|
| 217 |
+
font=dict(color='white')
|
| 218 |
+
)
|
| 219 |
+
return fig
|
| 220 |
+
|
| 221 |
+
def create_language_chart():
|
| 222 |
+
"""Create bar chart of datasets by language"""
|
| 223 |
+
df = create_dataframe()
|
| 224 |
+
language_counts = df['language'].value_counts().head(10)
|
| 225 |
+
|
| 226 |
+
fig = px.bar(
|
| 227 |
+
x=language_counts.index,
|
| 228 |
+
y=language_counts.values,
|
| 229 |
+
title='Top 10 Languages',
|
| 230 |
+
labels={'x': 'Language', 'y': 'Number of Datasets'},
|
| 231 |
+
color=language_counts.values,
|
| 232 |
+
color_continuous_scale='Viridis'
|
| 233 |
+
)
|
| 234 |
+
fig.update_layout(
|
| 235 |
+
paper_bgcolor='rgba(0,0,0,0)',
|
| 236 |
+
plot_bgcolor='rgba(0,0,0,0)',
|
| 237 |
+
font=dict(color='white'),
|
| 238 |
+
showlegend=False
|
| 239 |
+
)
|
| 240 |
+
return fig
|
| 241 |
+
|
| 242 |
+
def create_downloads_chart():
|
| 243 |
+
"""Create bar chart of top datasets by downloads"""
|
| 244 |
+
df = create_dataframe()
|
| 245 |
+
top_downloads = df.nlargest(15, 'downloads')[['name', 'downloads']]
|
| 246 |
+
top_downloads['short_name'] = top_downloads['name'].apply(lambda x: x.split('/')[-1][:30])
|
| 247 |
+
|
| 248 |
+
fig = px.bar(
|
| 249 |
+
top_downloads,
|
| 250 |
+
x='downloads',
|
| 251 |
+
y='short_name',
|
| 252 |
+
orientation='h',
|
| 253 |
+
title='Top 15 Datasets by Downloads',
|
| 254 |
+
labels={'downloads': 'Downloads', 'short_name': 'Dataset'},
|
| 255 |
+
color='downloads',
|
| 256 |
+
color_continuous_scale='Plasma'
|
| 257 |
+
)
|
| 258 |
+
fig.update_layout(
|
| 259 |
+
paper_bgcolor='rgba(0,0,0,0)',
|
| 260 |
+
plot_bgcolor='rgba(0,0,0,0)',
|
| 261 |
+
font=dict(color='white'),
|
| 262 |
+
height=600,
|
| 263 |
+
showlegend=False
|
| 264 |
+
)
|
| 265 |
+
return fig
|
| 266 |
+
|
| 267 |
+
def create_size_distribution_chart():
|
| 268 |
+
"""Create distribution chart of dataset sizes"""
|
| 269 |
+
df = create_dataframe()
|
| 270 |
+
size_counts = df['size'].value_counts()
|
| 271 |
+
|
| 272 |
+
fig = px.bar(
|
| 273 |
+
x=size_counts.index,
|
| 274 |
+
y=size_counts.values,
|
| 275 |
+
title='Dataset Size Distribution',
|
| 276 |
+
labels={'x': 'Size Category', 'y': 'Number of Datasets'},
|
| 277 |
+
color=size_counts.values,
|
| 278 |
+
color_continuous_scale='Cividis'
|
| 279 |
+
)
|
| 280 |
+
fig.update_layout(
|
| 281 |
+
paper_bgcolor='rgba(0,0,0,0)',
|
| 282 |
+
plot_bgcolor='rgba(0,0,0,0)',
|
| 283 |
+
font=dict(color='white'),
|
| 284 |
+
showlegend=False
|
| 285 |
+
)
|
| 286 |
+
return fig
|
| 287 |
+
|
| 288 |
+
def export_to_csv(keyword, language, category, min_downloads, min_likes):
|
| 289 |
+
"""Export filtered datasets to CSV"""
|
| 290 |
+
df = filter_datasets(keyword, language, category, min_downloads, min_likes)
|
| 291 |
+
output_path = "/tmp/cybersecurity_datasets.csv"
|
| 292 |
+
df.to_csv(output_path, index=False)
|
| 293 |
+
return output_path
|
| 294 |
+
|
| 295 |
+
def export_to_json(keyword, language, category, min_downloads, min_likes):
|
| 296 |
+
"""Export filtered datasets to JSON"""
|
| 297 |
+
df = filter_datasets(keyword, language, category, min_downloads, min_likes)
|
| 298 |
+
output_path = "/tmp/cybersecurity_datasets.json"
|
| 299 |
+
df.to_json(output_path, orient='records', indent=2)
|
| 300 |
+
return output_path
|
| 301 |
+
|
| 302 |
+
# Create Gradio interface
|
| 303 |
+
with gr.Blocks(theme=gr.themes.Soft(primary_hue="blue", secondary_hue="purple").set(
|
| 304 |
+
body_background_fill='*primary_900',
|
| 305 |
+
body_background_fill_dark='*primary_950',
|
| 306 |
+
block_background_fill='*primary_800',
|
| 307 |
+
block_background_fill_dark='*primary_900',
|
| 308 |
+
block_border_color='*primary_600',
|
| 309 |
+
input_background_fill='*primary_700',
|
| 310 |
+
button_primary_background_fill='*primary_600',
|
| 311 |
+
button_primary_background_fill_hover='*primary_500',
|
| 312 |
+
)) as demo:
|
| 313 |
+
|
| 314 |
+
gr.Markdown("""
|
| 315 |
+
# π Cybersecurity Dataset Explorer
|
| 316 |
+
|
| 317 |
+
Explore and analyze 80+ cybersecurity datasets from HuggingFace
|
| 318 |
+
|
| 319 |
+
**Features:**
|
| 320 |
+
- Search by keyword, language, category
|
| 321 |
+
- View detailed statistics and visualizations
|
| 322 |
+
- Export datasets metadata to CSV/JSON
|
| 323 |
+
- Preview dataset information
|
| 324 |
+
- Direct links to HuggingFace repositories
|
| 325 |
+
""")
|
| 326 |
+
|
| 327 |
+
# Statistics overview
|
| 328 |
+
with gr.Row():
|
| 329 |
+
stats = get_dataset_stats()
|
| 330 |
+
for key, value in stats.items():
|
| 331 |
+
gr.Markdown(f"### {key}\n## {value}")
|
| 332 |
+
|
| 333 |
+
# Main tabs
|
| 334 |
+
with gr.Tabs():
|
| 335 |
+
|
| 336 |
+
# Search & Filter Tab
|
| 337 |
+
with gr.Tab("π Search & Filter"):
|
| 338 |
+
with gr.Row():
|
| 339 |
+
with gr.Column(scale=1):
|
| 340 |
+
keyword_input = gr.Textbox(
|
| 341 |
+
label="Search Keyword",
|
| 342 |
+
placeholder="Enter keyword (name, description, or tags)...",
|
| 343 |
+
lines=1
|
| 344 |
+
)
|
| 345 |
+
language_dropdown = gr.Dropdown(
|
| 346 |
+
label="Language",
|
| 347 |
+
choices=["All"] + sorted(list(set([d['language'] for d in DATASETS_METADATA]))),
|
| 348 |
+
value="All"
|
| 349 |
+
)
|
| 350 |
+
category_dropdown = gr.Dropdown(
|
| 351 |
+
label="Category",
|
| 352 |
+
choices=["All", "ai", "defensive", "offensive", "compliance"],
|
| 353 |
+
value="All"
|
| 354 |
+
)
|
| 355 |
+
min_downloads_slider = gr.Slider(
|
| 356 |
+
label="Minimum Downloads",
|
| 357 |
+
minimum=0,
|
| 358 |
+
maximum=10000,
|
| 359 |
+
value=0,
|
| 360 |
+
step=100
|
| 361 |
+
)
|
| 362 |
+
min_likes_slider = gr.Slider(
|
| 363 |
+
label="Minimum Likes",
|
| 364 |
+
minimum=0,
|
| 365 |
+
maximum=100,
|
| 366 |
+
value=0,
|
| 367 |
+
step=1
|
| 368 |
+
)
|
| 369 |
+
search_btn = gr.Button("π Search Datasets", variant="primary")
|
| 370 |
+
|
| 371 |
+
with gr.Column(scale=3):
|
| 372 |
+
result_text = gr.Textbox(label="Search Results", lines=1)
|
| 373 |
+
results_table = gr.Dataframe(
|
| 374 |
+
label="Datasets",
|
| 375 |
+
wrap=True,
|
| 376 |
+
interactive=False
|
| 377 |
+
)
|
| 378 |
+
|
| 379 |
+
with gr.Row():
|
| 380 |
+
export_csv_btn = gr.Button("π₯ Export to CSV")
|
| 381 |
+
export_json_btn = gr.Button("π₯ Export to JSON")
|
| 382 |
+
|
| 383 |
+
with gr.Row():
|
| 384 |
+
csv_file = gr.File(label="CSV Download")
|
| 385 |
+
json_file = gr.File(label="JSON Download")
|
| 386 |
+
|
| 387 |
+
search_btn.click(
|
| 388 |
+
fn=search_datasets,
|
| 389 |
+
inputs=[keyword_input, language_dropdown, category_dropdown, min_downloads_slider, min_likes_slider],
|
| 390 |
+
outputs=[results_table, result_text]
|
| 391 |
+
)
|
| 392 |
+
|
| 393 |
+
export_csv_btn.click(
|
| 394 |
+
fn=export_to_csv,
|
| 395 |
+
inputs=[keyword_input, language_dropdown, category_dropdown, min_downloads_slider, min_likes_slider],
|
| 396 |
+
outputs=csv_file
|
| 397 |
+
)
|
| 398 |
+
|
| 399 |
+
export_json_btn.click(
|
| 400 |
+
fn=export_to_json,
|
| 401 |
+
inputs=[keyword_input, language_dropdown, category_dropdown, min_downloads_slider, min_likes_slider],
|
| 402 |
+
outputs=json_file
|
| 403 |
+
)
|
| 404 |
+
|
| 405 |
+
# Dataset Details Tab
|
| 406 |
+
with gr.Tab("π Dataset Details"):
|
| 407 |
+
dataset_selector = gr.Dropdown(
|
| 408 |
+
label="Select Dataset",
|
| 409 |
+
choices=[d['name'] for d in DATASETS_METADATA],
|
| 410 |
+
value=DATASETS_METADATA[0]['name'] if DATASETS_METADATA else None
|
| 411 |
+
)
|
| 412 |
+
view_details_btn = gr.Button("View Details", variant="primary")
|
| 413 |
+
|
| 414 |
+
dataset_details = gr.Markdown(label="Dataset Information")
|
| 415 |
+
preview_table = gr.Dataframe(label="Preview (Mock Data)")
|
| 416 |
+
dataset_link = gr.Textbox(label="HuggingFace URL")
|
| 417 |
+
|
| 418 |
+
view_details_btn.click(
|
| 419 |
+
fn=get_dataset_details,
|
| 420 |
+
inputs=dataset_selector,
|
| 421 |
+
outputs=[dataset_details, preview_table, dataset_link]
|
| 422 |
+
)
|
| 423 |
+
|
| 424 |
+
# Statistics & Visualizations Tab
|
| 425 |
+
with gr.Tab("π Statistics & Charts"):
|
| 426 |
+
gr.Markdown("## Dataset Analytics Dashboard")
|
| 427 |
+
|
| 428 |
+
with gr.Row():
|
| 429 |
+
category_chart = gr.Plot(label="Category Distribution")
|
| 430 |
+
language_chart = gr.Plot(label="Language Distribution")
|
| 431 |
+
|
| 432 |
+
with gr.Row():
|
| 433 |
+
downloads_chart = gr.Plot(label="Top Downloads")
|
| 434 |
+
|
| 435 |
+
with gr.Row():
|
| 436 |
+
size_chart = gr.Plot(label="Size Distribution")
|
| 437 |
+
|
| 438 |
+
refresh_charts_btn = gr.Button("π Refresh Charts", variant="primary")
|
| 439 |
+
|
| 440 |
+
def refresh_all_charts():
|
| 441 |
+
return (
|
| 442 |
+
create_category_chart(),
|
| 443 |
+
create_language_chart(),
|
| 444 |
+
create_downloads_chart(),
|
| 445 |
+
create_size_distribution_chart()
|
| 446 |
+
)
|
| 447 |
+
|
| 448 |
+
refresh_charts_btn.click(
|
| 449 |
+
fn=refresh_all_charts,
|
| 450 |
+
outputs=[category_chart, language_chart, downloads_chart, size_chart]
|
| 451 |
+
)
|
| 452 |
+
|
| 453 |
+
# Load charts on startup
|
| 454 |
+
demo.load(
|
| 455 |
+
fn=refresh_all_charts,
|
| 456 |
+
outputs=[category_chart, language_chart, downloads_chart, size_chart]
|
| 457 |
+
)
|
| 458 |
+
|
| 459 |
+
# About Tab
|
| 460 |
+
with gr.Tab("βΉοΈ About"):
|
| 461 |
+
gr.Markdown("""
|
| 462 |
+
## About Dataset Explorer
|
| 463 |
+
|
| 464 |
+
This application provides a comprehensive interface to explore 80 cybersecurity datasets from HuggingFace.
|
| 465 |
+
|
| 466 |
+
### Features:
|
| 467 |
+
|
| 468 |
+
1. **Search & Filter**: Find datasets by keyword, language, category, popularity
|
| 469 |
+
2. **Dataset Details**: View comprehensive information about each dataset
|
| 470 |
+
3. **Statistics**: Visual analytics with interactive charts
|
| 471 |
+
4. **Export**: Download filtered results as CSV or JSON
|
| 472 |
+
5. **Direct Links**: Access to HuggingFace repositories
|
| 473 |
+
|
| 474 |
+
### Categories:
|
| 475 |
+
|
| 476 |
+
- **AI**: Datasets for training AI/ML models
|
| 477 |
+
- **Defensive**: Blue team, threat detection, incident response
|
| 478 |
+
- **Offensive**: Red team, penetration testing, exploits
|
| 479 |
+
- **Compliance**: NIST, ISO 27001, regulatory frameworks
|
| 480 |
+
|
| 481 |
+
### Data Sources:
|
| 482 |
+
|
| 483 |
+
All datasets are publicly available on HuggingFace Hub. This explorer provides
|
| 484 |
+
metadata and filtering capabilities. To access the actual dataset content,
|
| 485 |
+
click the HuggingFace URL for any dataset.
|
| 486 |
+
|
| 487 |
+
### Technologies:
|
| 488 |
+
|
| 489 |
+
- **Gradio**: Interactive web interface
|
| 490 |
+
- **Pandas**: Data manipulation
|
| 491 |
+
- **Plotly**: Interactive visualizations
|
| 492 |
+
- **HuggingFace Datasets**: Dataset metadata
|
| 493 |
+
|
| 494 |
+
---
|
| 495 |
+
|
| 496 |
+
**Created by:** AYI-NEDJIMI
|
| 497 |
+
**Version:** 1.0
|
| 498 |
+
**Last Updated:** February 2026
|
| 499 |
+
""")
|
| 500 |
+
|
| 501 |
+
# Footer
|
| 502 |
+
gr.Markdown("""
|
| 503 |
+
---
|
| 504 |
+
π‘ **Tip**: Use the search feature to find datasets by specific topics like "NIST", "penetration testing", "threat intelligence", etc.
|
| 505 |
+
""")
|
| 506 |
+
|
| 507 |
+
if __name__ == "__main__":
|
| 508 |
+
demo.launch()
|
requirements.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
gradio==4.44.1
|
| 2 |
+
pandas==2.1.4
|
| 3 |
+
plotly==5.18.0
|
| 4 |
+
datasets==2.16.1
|