'gpt-2' 검색 결과

Learn how GPT language models work by preparing text, training a small model, and generating samples. Covers CPU experiments, custom datasets, further training of pretrained GPT-2 models, and reproducing GPT-2 with multiple GPUs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train a small model on Shakespeare using a CPU
• Prepare custom text for model training
• Further train a pretrained GPT-2 model

Learn how GPT language models work by preparing text, training a small model, and generating samples. Covers CPU experiments, custom datasets, further training of pretrained GPT-2 models, and reproducing GPT-2 with multiple GPUs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train a small model on Shakespeare using a CPU
• Prepare custom text for model training
• Further train a pretrained GPT-2 model

Investigate how transformer language models produce predictions with TransformerLens. This skill guides inspection of intermediate values and attention patterns, plus experiments that replace internal values to test their effects and identify components involved in model behavior.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Inspecting attention patterns and intermediate values
• Testing output changes by replacing internal activations
• Analyzing attention head contributions to predictions

Investigate how transformer language models produce predictions with TransformerLens. This skill guides inspection of intermediate values and attention patterns, plus experiments that replace internal values to test their effects and identify components involved in model behavior.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Inspecting attention patterns and intermediate values
• Testing output changes by replacing internal activations
• Analyzing attention head contributions to predictions

Implementação educacional do GPT em ~300 linhas. Reproduz o GPT-2 (124M) em OpenWebText. Código limpo e hackeável para aprender transformers. Por Andrej Karpathy. Perfeito para entender a arquitetura do GPT do zero. Treine em Shakespeare (CPU) ou OpenWebText (multi-GPU).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

언어 모델 내부 반응을 소수의 특징으로 분해해 학습한 개념을 조사합니다. 사전 학습 SAE 분석, 자체 SAE 훈련, 특징을 통한 출력 조정과 재구성 품질 평가를 안내합니다.

• 사전 학습 SAE로 특징을 조사합니다
• 특정 모델 층을 대상으로 SAE를 훈련합니다
• 입력 문장별 특징 반응을 비교합니다

Work with Waypoint models to analyze microbiome samples, the communities of microbes in a sample. Convert taxonomic abundance tables, generate numerical sample representations, configure further training for prediction tasks, and evaluate models on the Compass benchmark.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Convert microbiome abundance tables into Waypoint inputs
• Generate sample vectors for clustering and visualization
• Configure further training with sample labels

Guides experiments that inspect and change internal model values using nnsight. Covers local PyTorch models and remote experiments through NDIF, including replacing intermediate values to test how particular layers affect a model’s predictions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Inspecting layer outputs and attention patterns
• Replacing internal values to compare predictions
• Running large-model experiments through NDIF

Investigate why a PyTorch model behaves a certain way by changing its internal values with pyvene. The skill guides experiments that swap or remove activations, train interventions, compare outputs, and save or share intervention configurations.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Locating model components that affect an output
• Swapping activations to test behavioral hypotheses
• Training interventions to investigate causal structure

Use SAELens to train and analyze sparse autoencoders, which separate neural network activity into features that may represent recognizable concepts. Explore pretrained features, evaluate reconstruction quality, and investigate how individual features influence language model outputs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Exploring features with a pretrained SAE
• Training and evaluating a custom SAE
• Studying overlapping concepts in model activity

Guides experiments that inspect and modify internal values in PyTorch models to study their effects on predictions. Covers local analysis and remote experiments through NDIF for models too large for local GPUs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Inspect internal values and attention patterns across layers
• Replace internal values to compare changes in predictions
• Run large-model experiments without local GPUs

Train GPT-2 scale models (~124M parameters) efficiently on a single GPU. Covers GPT-124M architecture, tokenized dataset loading (e.g., HuggingFace Hub shards), modern optimizers (Muon, AdamW), mixed precision training, and training loop implementation.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Guides multi-GPU pre-training: DDP, FSDP2, ZeRO, tensor/pipeline/expert parallelism, fp8/Muon. Use when scaling a run, training MoE, or reproducing GPT-2 on rented GPUs.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.