Skip to content
cccskills
FreePublished on GitHub

excel-large-file-processing-and-cleaning

Process multi-sheet Excel data by finding columns through keywords and counting specified values. Clean text fields to retain Chinese characters, then export the processed data as an Excel file with its size and download link.

OriginalShow the original description

读取多 sheet Excel 文件,动态识别目标列进行统计,并使用正则清洗文本字段提取中文字符,最终输出标准化 Excel 文件。

Works withExcel
See how to install

Good for

  • Finding a target column by a keyword in its name
  • Counting specified values across Excel sheets
  • Extracting Chinese characters from mixed text
  • Exporting cleaned data to Excel

Files included (1)

  • SKILL.md2.2 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Skill Steps

This sub-skill covers one capability of the Excel workflow. For reading/counting/Parquet optimization, see the parent workflow SKILL.md.

Step1 文本字段清洗,使用正则表达式提取纯中文字符(过滤数字、特殊符号等)。

import re

def extract_chinese(text):
    if pd.isna(text):
        return text
    # 仅保留 Unicode 中文字符范围
    chinese_chars = re.findall(r'[一-龥]', str(text))
    cleaned = ''.join(chinese_chars)
    return cleaned if cleaned else ''

clean_col = '目标清洗列' # 占位示例,如'收货人'
if clean_col in df.columns:
    df[clean_col] = df[clean_col].apply(extract_chinese)

Step2 动态模糊匹配列名,并统计该列中特定值的数量。

# 动态查找包含特定关键字的列
keyword = 'type'
target_val = 'varchar'
target_col = next((col for col in df.columns if keyword in str(col).lower()), None)

total_target_count = 0
details = []

if target_col is not None:
    # 忽略大小写和首尾空格进行匹配
    mask = df[target_col].astype(str).str.lower().str.strip() == target_val
    count = mask.sum()
    total_target_count += count
    
    if count > 0:
        details.append({
            'sheet': target_sheet,
            'target_count': count,
            'total_rows': len(df)
        })

print(f"{'='*50}")
print(f"匹配列 '{target_col}' 中值为 '{target_val}' 的总数: {total_target_count}")
print(f"{'='*50}")
for detail in details:
    print(f"  {detail['sheet']}: {detail['target_count']} 个匹配项 (共 {detail['total_rows']} 行)")

Step3 将清洗和处理后的数据保存为 Excel,并输出文件大小与下载链接。

output_path = "/mnt/data/cleaned_data_output.xlsx"
df.to_excel(output_path, index=False)

file_size = os.path.getsize(output_path)
print(f"清洗后的数据已保存至: {output_path}")
print(f"文件大小: {file_size} 字节")
# 生成标准下载链接格式
print(f"下载链接: sandbox:{output_path}")

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Compare counts and shares for two categories in Excel data. The skill cleans the selected columns, calculates differences and percentages, and produces a summary spreadsheet plus bar and pie charts.

  • Comparing counts for two categories
  • Visualizing category shares
  • Cleaning merged-cell data before counting
OpenSenseNova/SenseNova-Skills5.9KUpdated Oct 9, 2026

Analyze Excel data using custom categories, count distributions, and compare groups with charts. Score text using its length, technical terms, numbers, and selected complex concepts, then visualize difficulty levels and the relationship between text length and scores.

  • Summarize Excel records using custom categories
  • Compare category distributions across groups
  • Score and grade text using adjustable rules
OpenSenseNova/SenseNova-Skills5.9KUpdated Oct 9, 2026

Count categories in a chosen data column after removing missing values and placeholder labels. Calculate each category’s share and create high-resolution charts combining counts, percentages, pie charts, and horizontal bars for distribution reports.

  • Compare record counts across categories
  • Show category shares with pie and bar charts
  • Exclude placeholder labels before counting
OpenSenseNova/SenseNova-Skills5.9KUpdated Oct 9, 2026

Clean category and numeric fields in structured data, compare groups with counts and cross-tabulations, and produce charts, Excel results, and a Markdown report. Export visuals as high-resolution PNG and SVG files for use in reports.

  • Compare category distributions across groups
  • Clean category labels and fill merged-cell gaps
  • Export analysis results to Excel
OpenSenseNova/SenseNova-Skills5.9KUpdated Oct 9, 2026

Filter Excel data using candidate column names, RGB color values, text prefixes, and regular expressions, which match text patterns. Calculate totals and averages, then export results to Excel with highlighted rows and a download link.

  • Filtering rows by RGB values or identifier prefixes
  • Converting text to numbers and calculating totals or averages
  • Extracting matching commands with their identifiers and descriptions
OpenSenseNova/SenseNova-Skills5.9KUpdated Oct 9, 2026

Filter rows from Excel or previously converted Parquet files, a format designed for efficient data storage. Summarize matching records by group with counts, percentages, and totals, then export the summary to Excel.

  • Count records containing a specified risk level by work category
  • Calculate group shares among matching records
  • Summarize large datasets already converted to Parquet
OpenSenseNova/SenseNova-Skills5.9KUpdated Oct 9, 2026

See all skills by OpenSenseNova

Report a problem with this skill