machine-learning

어떤 스킬인가요?

Machine learning development patterns, model training, evaluation, and

설치 방법

프로젝트 폴더에서 아래 명령어를 실행하고, 설치할 에이전트를 선택하세요.

설치 명령을 직접 실행해야 적용됩니다. 지원 에이전트와 필요한 권한·라이선스는 제작자의 안내를 확인하세요.

스킬 원문 파일

---
author: Joseph OBrien
status: unpublished
updated: '2025-12-23'
version: 1.0.1
tag: skill
type: reference
parent: machine-learning
---

# Data Preprocessing Patterns

## Missing Value Strategies

### Numerical Features

```python
# Simple imputation
df['col'].fillna(df['col'].median(), inplace=True)

# Model-based imputation
from sklearn.impute import KNNImputer
imputer = KNNImputer(n_neighbors=5)
df_imputed = imputer.fit_transform(df)
```

### Categorical Features

```python
# Mode imputation
df['col'].fillna(df['col'].mode()[0], inplace=True)

# New category for missing
df['col'].fillna('MISSING', inplace=True)
```

## Feature Scaling

```python
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler

# Standard scaling (mean=0, std=1) - use for most models
scaler = StandardScaler()

# MinMax scaling [0,1] - use for neural networks
scaler = MinMaxScaler()

# Robust scaling - use when outliers present
scaler = RobustScaler()

X_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
```

## Categorical Encoding

```python
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
import category_encoders as ce

# One-hot encoding (low cardinality)
encoder = OneHotEncoder(sparse=False, handle_unknown='ignore')

# Target encoding (high cardinality)
encoder = ce.TargetEncoder(cols=['category_col'])

# Frequency encoding
df['col_freq'] = df['col'].map(df['col'].value_counts(normalize=True))
```

## Feature Engineering Examples

### Temporal Features

```python
df['hour'] = df['timestamp'].dt.hour
df['day_of_week'] = df['timestamp'].dt.dayofweek
df['is_weekend'] = df['day_of_week'].isin([5, 6]).astype(int)
df['month'] = df['timestamp'].dt.month

# Cyclical encoding
df['hour_sin'] = np.sin(2 * np.pi * df['hour'] / 24)
df['hour_cos'] = np.cos(2 * np.pi * df['hour'] / 24)
```

### Numerical Transformations

```python
# Log transform (right-skewed data)
df['col_log'] = np.log1p(df['col'])

# Box-Cox transform
from scipy.stats import boxcox
df['col_bc'], lambda_param = boxcox(df['col'] + 1)

# Polynomial features
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)
```

### Interaction Features

```python
# Manual interactions
df['feature_interaction'] = df['feat1'] * df['feat2']
df['feature_ratio'] = df['feat1'] / (df['feat2'] + 1e-8)

# Automated interactions
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=True)
```

## Outlier Handling

```python
# IQR method
Q1 = df['col'].quantile(0.25)
Q3 = df['col'].quantile(0.75)
IQR = Q3 - Q1
lower_bound = Q1 - 1.5 * IQR
upper_bound = Q3 + 1.5 * IQR
df_clean = df[(df['col'] >= lower_bound) & (df['col'] <= upper_bound)]

# Clipping
df['col_clipped'] = df['col'].clip(lower=lower_bound, upper=upper_bound)

# Z-score method
from scipy import stats
z_scores = np.abs(stats.zscore(df['col']))
df_clean = df[z_scores < 3]
```

## Data Leakage Prevention

**Common Leakage Sources:**

1. Future information in features
2. Target information in features
3. Train-test contamination during preprocessing

**Prevention:**

```python
# Always fit on train, transform on test
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Use pipelines
from sklearn.pipeline import Pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])
pipeline.fit(X_train, y_train)
```

{
  "schema_version": "2.0",
  "meta": {
    "generated_at": "2026-06-27T18:08:10.902Z",
    "slug": "89jobrien-machine-learning",
    "source_url": "https://github.com/89jobrien/steve/tree/main/steve/skills/machine-learning",
    "source_ref": "main",
    "model": "codex",
    "analysis_version": "3.0.0",
    "source_type": "community",
    "content_hash": "0d45a3b3720fbc923d62041401f7ce973b55d89faa31342eb79806b2941eb442",
    "tree_hash": "f9ce116825b50032fcd8db72d2fba0328d18d61d285841692fd69f659447765d"
  },
  "skill": {
    "name": "machine-learning",
    "description": "Machine learning development patterns, model training, evaluation, and deployment. Use when building ML pipelines, training models, feature engineering, model evaluation, or deploying ML systems to production.",
    "summary": "Machine learning development patterns, model training, evaluation, and deployment. Use when building...",
    "icon": "📦",
    "version": "1.0.1",
    "author": "Joseph OBrien",
    "license": "MIT",
    "category": "data",
    "tags": [
      "machine-learning",
      "mlops",
      "model-training",
      "feature-engineering",
      "model-evaluation"
    ],
    "supported_tools": [
      "claude",
      "codex",
      "claude-code"
    ],
    "risk_factors": []
  },
  "security_audit": {
    "risk_level": "low",
    "is_blocked": false,
    "safe_to_publish": true,
    "summary": "Static analysis reported external command, weak cryptography, and reconnaissance patterns. Manual review found instructional Markdown, Python preprocessing examples, and ML terminology, with no executable shell logic or malicious intent.",
    "risk_factor_evidence": [],
    "critical_findings": [],
    "high_findings": [],
    "medium_findings": [],
    "low_findings": [
      {
        "title": "False Positive: Python Examples Flagged as Shell Execution",
        "description": "The flagged references/preprocessing.md locations are fenced Python examples for imputation, scaling, encoding, feature engineering, outlier handling, and pipelines. No shell command execution or command interpolation is present.",
        "locations": [
          {
            "file": "references/preprocessing.md",
            "line_start": 17,
            "line_end": 25
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 29,
            "line_end": 35
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 39,
            "line_end": 53
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 57,
            "line_end": 69
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 75,
            "line_end": 84
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 88,
            "line_end": 100
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 104,
            "line_end": 112
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 116,
            "line_end": 132
          },
          {
            "file": "references/preprocessing.md",
            "line_start": 144,
            "line_end": 157
          }
        ],
        "confidence": 0.97,
        "confidence_reasoning": "The reviewed locations are Markdown code fences containing ordinary Python data science snippets. I found no subprocess, shell, network, or secret handling behavior in those examples."
      },
      {
        "title": "False Positive: Backtick Formatting Flagged as Commands",
        "description": "The SKILL.md locations use Markdown backticks for a model directory tree and reference file names. This is documentation formatting, not command execution.",
        "locations": [
          {
            "file": "SKILL.md",
            "line_start": 178,
            "line_end": 187
          },
          {
            "file": "SKILL.md",
            "line_start": 206,
            "line_end": 208
          }
        ],
        "confidence": 0.96,
        "confidence_reasoning": "The flagged text is inside a Markdown code block or inline Markdown formatting. It contains paths and file names, not runnable shell instructions."
      },
      {
        "title": "False Positive: Weak Cryptography Pattern",
        "description": "The SKILL.md description is plain metadata about machine learning workflows. No cryptographic algorithm, hash function, cipher, or credential handling appears at the flagged location.",
        "locations": [
          {
            "file": "SKILL.md",
            "line_start": 3,
            "line_end": 5
          }
        ],
        "confidence": 0.98,
        "confidence_reasoning": "The flagged location contains only the skill description text. The weak cryptography alert has no semantic support in the reviewed content."
      },
      {
        "title": "False Positive: System Reconnaissance Pattern",
        "description": "The flagged SKILL.md locations contain ML feature and tuning terms, including geospatial features and grid search. They do not request host, network, user, process, or environment reconnaissance.",
        "locations": [
          {
            "file": "SKILL.md",
            "line_start": 68,
            "line_end": 68
          },
          {
            "file": "SKILL.md",
            "line_start": 98,
            "line_end": 98
          }
        ],
        "confidence": 0.97,
        "confidence_reasoning": "The surrounding context is machine learning feature engineering and hyperparameter tuning. I found no commands or instructions for gathering system information."
      }
    ],
    "dangerous_patterns": [],
    "files_scanned": 2,
    "total_lines": 374,
    "audit_model": "codex",
    "audited_at": "2026-06-27T18:08:10.902Z",
    "risk_factors": []
  },
  "content": {
    "user_title": "Build Machine Learning Pipelines",
    "value_statement": "Machine learning projects often fail from weak data preparation, unclear metrics, and poor deployment planning. This skill guides Claude, Codex, and Claude Code through practical ML lifecycle decisions.",
    "seo_keywords": [
      "Claude",
      "Codex",
      "Claude Code",
      "machine learning",
      "ML pipelines",
      "feature engineering",
      "model evaluation",
      "MLOps",
      "model training",
      "data preprocessing"
    ],
    "actual_capabilities": [
      "Guides problem framing for classification, regression, clustering, ranking, and anomaly detection.",
      "Recommends evaluation metrics by ML problem type.",
      "Explains data preparation patterns for missing values, scaling, encoding, and leakage prevention.",
      "Outlines model selection options based on data size and domain.",
      "Covers deployment, monitoring, retraining, and experiment tracking practices."
    ],
    "limitations": [
      "Provides guidance and examples, not a complete runnable ML framework.",
      "Requires user datasets, constraints, and target metrics for specific recommendations.",
      "Does not include all referenced supplemental files in this package.",
      "Does not validate model fairness, privacy, or compliance requirements automatically."
    ],
    "use_cases": [
      {
        "title": "Plan a New Predictive Model",
        "description": "Define the problem type, success metrics, split strategy, and first model candidates before implementation starts.",
        "target_user": "Data scientist"
      },
      {
        "title": "Improve Feature Preparation",
        "description": "Select preprocessing, encoding, scaling, and leakage prevention methods for a tabular machine learning dataset.",
        "target_user": "ML engineer"
      },
      {
        "title": "Prepare a Model for Production",
        "description": "Review serving options, monitoring signals, retraining triggers, and versioning practices before deployment.",
        "target_user": "MLOps lead"
      }
    ],
    "prompt_templates": [
      {
        "title": "Choose Metrics",
        "scenario": "Beginner planning for a supervised learning task.",
        "prompt": "Use this machine learning skill to recommend primary and secondary metrics for my problem. Problem type: [type]. Business goal: [goal]. Error costs: [costs]."
      },
      {
        "title": "Design Preprocessing",
        "scenario": "Intermediate data preparation for a tabular dataset.",
        "prompt": "Use this skill to design a preprocessing plan. Dataset columns: [columns]. Missing data: [patterns]. Categorical cardinality: [details]. Leakage risks: [risks]."
      },
      {
        "title": "Select Models",
        "scenario": "Intermediate model selection before experiments.",
        "prompt": "Use this machine learning skill to compare model families for my dataset. Rows: [count]. Features: [types]. Latency target: [target]. Interpretability needs: [needs]."
      },
      {
        "title": "Review Production Readiness",
        "scenario": "Advanced MLOps review before launch.",
        "prompt": "Use this skill to review production readiness for my ML system. Serving mode: [mode]. Monitoring plan: [plan]. Retraining policy: [policy]. Rollback process: [process]."
      }
    ],
    "output_examples": [
      {
        "input": "Plan metrics for a binary fraud model with rare positives.",
        "output": "Recommended output includes PR-AUC as the primary metric, recall at fixed precision, threshold tuning guidance, and a held-out test plan."
      },
      {
        "input": "Design preprocessing for mixed numerical and categorical customer data.",
        "output": [
          "Missing value strategy by feature type.",
          "Encoding choices for low and high cardinality columns.",
          "Scaling guidance for model families.",
          "Leakage checks before train and test transformation."
        ]
      },
      {
        "input": "Prepare a trained model for production deployment.",
        "output": "Recommended output covers serving pattern, latency targets, versioned artifacts, monitoring signals, retraining triggers, and rollback expectations."
      }
    ],
    "best_practices": [
      "Start with the problem type, target metric, and error costs before choosing algorithms.",
      "Fit preprocessing only on training data, then transform validation and test data.",
      "Track code, data, parameters, metrics, artifacts, and environment for every experiment."
    ],
    "anti_patterns": [
      "Do not tune models on the final test set.",
      "Do not use random splits for time-dependent data without checking leakage.",
      "Do not deploy a model without monitoring data drift, performance, and latency."
    ],
    "faq": [
      {
        "question": "What does this skill help with?",
        "answer": "It helps plan, build, evaluate, deploy, and monitor machine learning workflows."
      },
      {
        "question": "Can it train models by itself?",
        "answer": "No. It provides guidance and examples that an AI coding tool can apply in your project."
      },
      {
        "question": "Which AI tools are supported?",
        "answer": "The report lists support for Claude, Codex, and Claude Code."
      },
      {
        "question": "Does it include preprocessing examples?",
        "answer": "Yes. It includes examples for imputation, scaling, encoding, transformations, outlier handling, and leakage prevention."
      },
      {
        "question": "Is this only for deep learning?",
        "answer": "No. It covers classical ML, gradient boosting, neural networks, NLP, vision, time series, and production patterns."
      },
      {
        "question": "What should I provide in prompts?",
        "answer": "Provide the problem type, dataset shape, target metric, constraints, deployment needs, and known risks."
      }
    ]
  },
  "file_structure": [
    {
      "name": "references",
      "type": "dir",
      "path": "references",
      "children": [
        {
          "name": "preprocessing.md",
          "type": "file",
          "path": "references/preprocessing.md",
          "lines": 158
        }
      ]
    },
    {
      "name": "SKILL.md",
      "type": "file",
      "path": "SKILL.md",
      "lines": 216
    }
  ]
}

---
name: machine-learning
description: Machine learning development patterns, model training, evaluation, and
  deployment. Use when building ML pipelines, training models, feature engineering,
  model evaluation, or deploying ML systems to production.
author: Joseph OBrien
status: unpublished
updated: '2025-12-23'
version: 1.0.1
tag: skill
type: skill
---

# Machine Learning

Comprehensive machine learning skill covering the full ML lifecycle from experimentation to production deployment.

## When to Use This Skill

- Building machine learning pipelines
- Feature engineering and data preprocessing
- Model training, evaluation, and selection
- Hyperparameter tuning and optimization
- Model deployment and serving
- ML experiment tracking and versioning
- Production ML monitoring and maintenance

## ML Development Lifecycle

### 1. Problem Definition

**Classification Types:**

- Binary classification (spam/not spam)
- Multi-class classification (image categories)
- Multi-label classification (document tags)
- Regression (price prediction)
- Clustering (customer segmentation)
- Ranking (search results)
- Anomaly detection (fraud detection)

**Success Metrics by Problem Type:**

| Problem Type | Primary Metrics | Secondary Metrics |
|--------------|-----------------|-------------------|
| Binary Classification | AUC-ROC, F1 | Precision, Recall, PR-AUC |
| Multi-class | Macro F1, Accuracy | Per-class metrics |
| Regression | RMSE, MAE | R², MAPE |
| Ranking | NDCG, MAP | MRR |
| Clustering | Silhouette, Calinski-Harabasz | Davies-Bouldin |

### 2. Data Preparation

**Data Quality Checks:**

- Missing value analysis and imputation strategies
- Outlier detection and handling
- Data type validation
- Distribution analysis
- Target leakage detection

**Feature Engineering Patterns:**

- Numerical: scaling, binning, log transforms, polynomial features
- Categorical: one-hot, target encoding, frequency encoding, embeddings
- Temporal: lag features, rolling statistics, cyclical encoding
- Text: TF-IDF, word embeddings, transformer embeddings
- Geospatial: distance features, clustering, grid encoding

**Train/Test Split Strategies:**

- Random split (standard)
- Stratified split (imbalanced classes)
- Time-based split (temporal data)
- Group split (prevent data leakage)
- K-fold cross-validation

### 3. Model Selection

**Algorithm Selection Guide:**

| Data Size | Problem | Recommended Models |
|-----------|---------|-------------------|
| Small (<10K) | Classification | Logistic Regression, SVM, Random Forest |
| Small (<10K) | Regression | Linear Regression, Ridge, SVR |
| Medium (10K-1M) | Classification | XGBoost, LightGBM, Neural Networks |
| Medium (10K-1M) | Regression | XGBoost, LightGBM, Neural Networks |
| Large (>1M) | Any | Deep Learning, Distributed training |
| Tabular | Any | Gradient Boosting (XGBoost, LightGBM, CatBoost) |
| Images | Classification | CNN, ResNet, EfficientNet, Vision Transformers |
| Text | NLP | Transformers (BERT, RoBERTa, GPT) |
| Sequential | Time Series | LSTM, Transformer, Prophet |

### 4. Model Training

**Hyperparameter Tuning:**

- Grid Search: exhaustive, good for small spaces
- Random Search: efficient, good for large spaces
- Bayesian Optimization: smart exploration (Optuna, Hyperopt)
- Early stopping: prevent overfitting

**Common Hyperparameters:**

| Model | Key Parameters |
|-------|---------------|
| XGBoost | learning_rate, max_depth, n_estimators, subsample |
| LightGBM | num_leaves, learning_rate, n_estimators, feature_fraction |
| Random Forest | n_estimators, max_depth, min_samples_split |
| Neural Networks | learning_rate, batch_size, layers, dropout |

### 5. Model Evaluation

**Evaluation Best Practices:**

- Always use held-out test set for final evaluation
- Use cross-validation during development
- Check for overfitting (train vs validation gap)
- Evaluate on multiple metrics
- Analyze errors qualitatively

**Handling Imbalanced Data:**

- Resampling: SMOTE, undersampling
- Class weights: weighted loss functions
- Threshold tuning: optimize decision threshold
- Evaluation: use PR-AUC over ROC-AUC

### 6. Production Deployment

**Model Serving Patterns:**

- REST API (Flask, FastAPI, TF Serving)
- Batch inference (scheduled jobs)
- Streaming (real-time predictions)
- Edge deployment (mobile, IoT)

**Production Considerations:**

- Latency requirements (p50, p95, p99)
- Throughput (requests per second)
- Model size and memory footprint
- Fallback strategies
- A/B testing framework

### 7. Monitoring & Maintenance

**What to Monitor:**

- Prediction latency
- Input feature distributions (data drift)
- Prediction distributions (concept drift)
- Model performance metrics
- Error rates and types

**Retraining Triggers:**

- Performance degradation below threshold
- Significant data drift detected
- Scheduled retraining (daily, weekly)
- New training data available

## MLOps Best Practices

### Experiment Tracking

Track for every experiment:

- Code version (git commit)
- Data version (hash or version ID)
- Hyperparameters
- Metrics (train, validation, test)
- Model artifacts
- Environment (packages, versions)

### Model Versioning

```
models/
├── model_v1.0.0/
│   ├── model.pkl
│   ├── metadata.json
│   ├── requirements.txt
│   └── metrics.json
├── model_v1.1.0/
└── model_v2.0.0/
```

### CI/CD for ML

1. **Continuous Integration:**
   - Data validation tests
   - Model training tests
   - Performance regression tests

2. **Continuous Deployment:**
   - Staging environment validation
   - Shadow mode testing
   - Gradual rollout (canary)
   - Automatic rollback

## Reference Files

For detailed patterns and code examples, load reference files as needed:

- **`references/preprocessing.md`** - Data preprocessing patterns and feature engineering techniques
- **`references/model_patterns.md`** - Model architecture patterns and implementation examples
- **`references/evaluation.md`** - Comprehensive evaluation strategies and metrics

## Integration with Other Skills

- **performance** - For optimizing inference latency
- **testing** - For ML-specific testing patterns
- **database-optimization** - For feature store queries
- **debugging** - For model debugging and error analysis