colinalexander/super-skills · GitHub
Design, analyze, train, evaluate, and operationalize statistical or machine-learning systems. Use for data quality, experiments, causal inference, predictive modeling, deep learning, computer vision, fine-tuning, distributed training, or model monitoring; use application engineering for ordinary product architecture without an analytical or learned-model decision.
프로젝트 폴더에서 아래 명령어를 실행하고, 설치할 에이전트를 선택하세요.
npx skills add colinalexander/super-skills --skill data-science-and-ml설치 명령을 직접 실행해야 적용됩니다. 지원 에이전트와 필요한 권한·라이선스는 제작자의 안내를 확인하세요.
agents/openai.yamlinterface: display_name: "Data Science and Machine Learning" short_description: "Build valid, reproducible data and ML systems" default_prompt: "Use $data-science-and-ml to frame, validate, and operationalize this analytical or machine-learning task." policy: allow_implicit_invocation: true
references/data-and-experiments.md# Data and experiments ## Audit the data-generating process Document sources, collection rules, units, timestamps, joins, labels, exclusions, consent, and retention. Check schema and range validity, missingness mechanisms, duplication, survivorship, delayed outcomes, selection bias, and target leakage. Preserve a raw immutable layer when feasible and make transformations reproducible. ## Match inference to design For descriptive work, define denominators, cohorts, and uncertainty. For causal questions, state the treatment, outcome, estimand, assignment mechanism, interference assumptions, and plausible confounders. Prefer randomized assignment when ethical and practical; otherwise explain what assumptions identification requires. Predefine experiment population, randomization unit, exposure, primary metric, guardrails, power assumptions, duration, and analysis plan. Account for repeated looks, multiple outcomes, noncompliance, attrition, and novelty effects. Statistical significance does not establish material value. Communicate effect sizes and intervals alongside assumptions. Separate exploratory findings from confirmatory evidence.
references/deployment-and-monitoring.md# Deployment and monitoring ## Define the serving contract Specify inputs, outputs, preprocessing, model and feature versions, latency and resource budgets, concurrency, failure behavior, fallback, privacy, and human oversight. Keep training-serving transformations aligned and test the packaged artifact in a production-like path. Roll out through offline checks, shadow or replay evaluation where appropriate, a bounded canary, and explicit promotion criteria. Separate model quality from pipeline availability and product impact. ## Observe changing behavior Monitor input validity, missing features, distribution shift, prediction or generation quality, calibration where applicable, slice performance, latency, resource use, failures, and downstream outcomes. Define alert thresholds and owners before launch. Ground-truth delay can make immediate quality monitoring impossible; use proxies cautiously and backfill outcome evaluation when labels arrive. Log enough to investigate behavior without retaining unnecessary sensitive data. Define retraining triggers, approval and validation gates, rollback, audit history, and retirement conditions. Automated retraining must not silently promote a model solely because a pipeline completed.
references/modeling-and-evaluation.md# Modeling and evaluation ## Establish a credible baseline Start with a rule, historical rate, simple statistical model, or current production system. Split data by the boundary the model will face—often time, entity, site, or geography rather than a random row split. Keep the final test set isolated from model and threshold selection. Choose metrics from operational error costs. Calibration, ranking, threshold behavior, latency, memory, and slice performance may matter more than a single aggregate score. Compare uncertainty, not just point estimates. ## Diagnose before increasing complexity Inspect label quality, feature leakage, residuals, confusion patterns, difficult slices, outliers, and errors under distribution shift. Use ablations to determine which data or components create value. Tune against validation data with a recorded search budget; never tune on the test set. For vision and other perception systems, include capture conditions, annotation policy, geometry where relevant, and edge deployment constraints. For generative systems, define task-specific quality, safety, factuality, robustness, and human-review protocols rather than relying on one automatic metric. Produce a model card or equivalent record covering intended use, excluded use, data, evaluation, limitations, and known failure modes.
references/training-and-scale.md# Training and scale ## Reproduce the run Track dataset identity and splits, preprocessing, code revision, environment, configuration, seeds, hardware, checkpoints, and metrics. Deterministic execution may have performance costs and platform limits; state the reproducibility level actually achieved. Validate the single-device training loop before distributing it. Confirm tensor shapes, loss behavior, gradient flow, evaluation mode, checkpoint restoration, and a small overfit test. Profile memory, compute, input, and communication before choosing an optimization. ## Choose scale from the bottleneck Use mixed precision, accumulation, activation checkpointing, sharding, offload, quantization, or distributed data/model parallelism only when their tradeoffs match measured constraints. Define checkpoint format, world-size portability, failure recovery, and numerical validation. For parameter-efficient fine-tuning, compare against prompting, retrieval, and full fine-tuning. Specify base model, data rights and quality, adapter method, target modules, evaluation, merge or multi-adapter serving strategy, and rollback. Reduced trainable parameters do not remove data, safety, or evaluation obligations. Verify all version-sensitive framework and accelerator APIs against primary documentation for the installed environment.
SKILL.md--- name: data-science-and-ml description: Design, analyze, train, evaluate, and operationalize statistical or machine-learning systems. Use for data quality, experiments, causal inference, predictive modeling, deep learning, computer vision, fine-tuning, distributed training, or model monitoring; use application engineering for ordinary product architecture without an analytical or learned-model decision. --- # Data Science and Machine Learning Make the decision, estimand, data-generating process, and validation boundary explicit before selecting a method or model. Analytical validity outranks algorithm novelty. ## Frame the problem Define the decision or product behavior, unit of analysis, target or estimand, population, time horizon, cost of errors, operational constraints, and baseline. Determine whether the task is descriptive, predictive, causal, ranking, generation, detection, or optimization. Load the relevant reference: - data quality, statistics, experiments, and causal questions: [data-and-experiments.md](references/data-and-experiments.md); - features, models, validation, and error analysis: [modeling-and-evaluation.md](references/modeling-and-evaluation.md); - reproducible training, accelerators, fine-tuning, and distributed scale: [training-and-scale.md](references/training-and-scale.md); - serving, monitoring, governance, and retraining: [deployment-and-monitoring.md](references/deployment-and-monitoring.md). ## Build an evidence chain 1. Trace data provenance, collection, exclusions, labels, and permissions. 2. Detect leakage, confounding, selection effects, missingness, imbalance, and temporal drift. 3. Establish a simple, credible baseline and a validation design that matches deployment. 4. Choose metrics from decision costs and inspect performance across relevant slices. 5. Track data, code, configuration, environment, randomness, and artifacts for reproducibility. 6. Perform error analysis before adding complexity. 7. Define deployment, fallback, monitoring, and retirement conditions. Report uncertainty and limitations. Do not treat correlation as causation, test-set iteration as validation, or benchmark gains as production value. ## Respect boundaries `application-engineering` owns surrounding APIs and product architecture. `software-delivery` owns the code-change workflow. This skill owns the validity and behavior of analytical and learned systems. Check current primary documentation before using version-sensitive framework, accelerator, or model APIs.