Results for “pandas”

Build Python workflows for filtering, grouping, joining, and reshaping table data with Polars. Get guidance on migrating pandas code and using lazy evaluation, which optimizes a processing plan before execution, for large datasets.

• Filtering and summarizing large CSV files
• Migrating pandas processing code to Polars
• Joining and reshaping multiple tables

Clean, combine, summarize, and reshape tabular data with pandas DataFrames, Python's table-like data structures. Produces transformation code and validation checks, including time series processing and memory-conscious approaches for large datasets.

• Fill missing values and correct data types
• Combine tables using multiple matching columns
• Create grouped summaries and pivot tables

Optimize scientific Python workloads for NVIDIA GPUs by choosing suitable libraries and reducing bottlenecks. Compare outputs with the CPU version and measure complete processing time, including data transfers, to determine whether the GPU implementation provides a useful improvement.

• Assess whether slow Python processing would benefit from a GPU
• Move NumPy or pandas workloads to suitable GPU libraries
• Validate GPU results against CPU outputs

Work with geographic points, lines, and areas in Python using GeoPandas. Guidance and local audit tools help check coordinate systems, geometry validity, spatial join matches, and export settings before trusting or sharing analysis results.

• Checking geographic data before analysis
• Reviewing coordinate systems and transformations
• Auditing matches between points and areas

Create static statistical charts from pandas tables to explore distributions, relationships, and group differences. Supports scatter plots, regression displays, pair plots, and heatmaps, with explicit choices for aggregation, uncertainty, missing data, and visual styling.

• Exploring distributions and relationships in tabular data
• Comparing group differences with box or violin plots
• Visualizing correlation matrices as heatmaps

Recognize classic candlestick patterns using pandas, a Python library for tabular data. Combine bullish and bearish pattern scores into long, short, or stand-aside signals, with adjustable thresholds for candle bodies and shadows.

• Detect classic patterns in candlestick data
• Combine bullish and bearish patterns into one signal
• Adjust thresholds for candle bodies and shadows

Generate trading signals using Ichimoku Kinko Hyo, a five-line chart framework. Combine conversion-line and base-line crossovers with price position and cloud direction to classify conditions as long, short, or stand aside using pandas.

• Evaluate trends with the five Ichimoku lines
• Identify crossovers that agree with the cloud
• Adjust calculation periods and displacement

Analyze price swings to detect Elliott Wave structures: five waves following a trend and three waves correcting it. Check wave rules and Fibonacci ratios, then generate signals for completed advances or pullbacks using a pandas implementation.

• Detecting five-wave advances in price data
• Identifying completed ABC corrections
• Adjusting wave-detection parameters

Analyze proxy, DNS, and firewall traffic logs to uncover unapproved SaaS and cloud services. Group usage by domain, compare it with approved applications, and produce a report with traffic volumes, user counts, risk scores, and remediation recommendations.

• Finding unapproved SaaS and cloud services
• Measuring traffic volumes and users by service
• Prioritizing services by potential data leakage risk

Analyze endpoint, cloud storage, and email logs for signs of insider data theft. Use pandas to compare upload volumes with normal user behavior and examine data loss prevention alerts, unusual file access, and activity outside working hours.

• Investigating suspected insider data theft
• Finding users with unusually large uploads
• Reviewing file access outside working hours

Run multi-line Python scripts through run_shell when execute_code_sandbox repeatedly fails. The guide explains heredocs, a way to pass blocks of text to a command, with examples for PDF creation, data processing, and inspecting errors.

• Running Python after repeated sandbox errors
• Creating a PDF with ReportLab
• Summarizing CSV data with pandas

Use when the user has tabular data (pandas DataFrame, parquet, csv, Arrow, json) and wants to filter, group, aggregate, join, or speed up slow pandas. Provides chDB DataStore - same pandas API, ClickHouse engine underneath. Also handles reading from S3, MySQL, PostgreSQL, MongoDB, ClickHouse Cloud, Iceberg, Delta Lake as DataFrames and joining across sources. TRIGGER when: user mentions DataFrame, parquet, csv, "fast pandas", "speed up pandas", or cross-source DataFrame joins; user imports `chdb.datastore` or `from datastore import DataStore`. SKIP this skill for raw SQL syntax (use chdb-sql instead), ClickHouse server administration, or non-Python DataStore API work.

The English summary is being prepared. Showing the original description.

Use Dask to process tables, numerical arrays, and file collections in pieces across CPU cores or machines. Helps choose data structures, size memory-friendly chunks, coordinate dependent tasks, and diagnose slow or memory-heavy workflows.

• Aggregate CSV or Parquet files that exceed available memory
• Process large scientific arrays in chunks
• Clean JSON records and logs before analysis

Generate Python code for Dataverse SDK workflows, including retries, bulk data changes, efficient queries, table metadata, and large file uploads. The skill also covers pandas integration and asks for type hints, docstrings, and official API references.

• Implement retries for temporary Dataverse errors
• Create bulk data operations and efficient queries
• Manage custom tables and metadata caches

Examine Zeek connection logs for regular callbacks that may indicate communication with an attacker’s control server. Use ZAT and Pandas to group connections, measure timing variation, and identify source and destination pairs for further investigation.

• Hunting for periodic callbacks in Zeek connection logs
• Comparing connection timing across source and destination pairs
• Developing beaconing detection rules and hunting queries

Analyze access logs from AWS API Gateway, Kong, and Nginx with pandas, a Python data analysis library. Examine resource access patterns, authentication failures, and suspicious requests to investigate API abuse and inform detection rules.

• Investigating sequential access to resource IDs
• Examining spikes in authentication failures
• Checking for rate limit bypass and injection attempts

Extract data from Excel sheets with misplaced headers, merged header cells, or introductory rows. Uses text patterns to locate headers, clean column names, and remove empty rows so the data can be processed with pandas.

• Extracting tables beneath title and metadata rows
• Reading sheets with inconsistent header positions
• Cleaning blank or duplicate column names

Helps explore datasets, clean missing or inconsistent values, and explain patterns through statistics and charts. Uses Python and SQL to compare groups, examine relationships, and present findings in plain language with documented analysis steps.

• Checking missing values, duplicates, and inconsistent formats
• Exploring trends and distributions with charts
• Comparing groups with statistical tests

Process Excel workbooks by saving Python scripts to files and running them through the shell. Covers multiple sheets, data transformations, formulas, formatting, and error reporting for complex spreadsheet tasks.

• Transform revenue data and save a processed Excel workbook
• Apply calculations across multiple worksheets
• Format report headers with colors and alignment

Process complex Excel files by running Python scripts directly through run_shell. Use pandas for data transformations and openpyxl for workbook editing, including multiple sheets, formulas, and formatting, with explicit saving and error reporting.

• Processing multiple Excel sheets in one script
• Calculating new columns from spreadsheet data
• Formatting report headers and cells

Validate spreadsheet sources before running Python scripts to transform data, update multiple sheets, or apply formatting. Check required data first, try backup sources when needed, and report errors clearly if processing cannot proceed.

• Checking required columns before processing Excel files
• Transforming data and saving a new workbook
• Updating formatting across multiple sheets

Diagnose Python spreadsheet failures through staged checks of the interpreter, library imports, file access, and basic operations. Minimal test scripts help separate environment problems from errors in the full processing script.

• Investigating unclear errors returned by run_shell
• Diagnosing spreadsheet scripts that suddenly stop working
• Checking Python setup in a new environment or container

Troubleshoot CSV and XLSX processing when execute_code_sandbox fails. The workflow locates input files, saves processing code as a Python script, captures execution errors, checks dependencies and file access, and confirms output exists before reading it.

• Recovering spreadsheet processing after sandbox errors
• Locating CSV and XLSX input files
• Diagnosing dependency, path, and encoding errors

在对具有多层索引的Pandas DataFrame进行分组计算时，确保返回的结果保留原始的完整索引结构，而不是仅保留分组键。

The English summary is being prepared. Showing the original description.