跳到正文
cccskills
免费在 GitHub 公开

invalid-data-cleaning

Prepare large Excel datasets by combining sheets, using Parquet, a format for efficient table storage, and cleaning a selected text column to retain only Chinese characters. Export the cleaned data as an Excel file with a local download link.

中文简介正在准备中。当前显示英语简介。

原文查看原文说明

用于大规模Excel数据的预处理,通过统计总行数判断是否转换为Parquet格式以提升读写效率,并使用正则表达式清洗指定文本列(如仅保留中文字符),最后导出清洗后的文件并提供下载链接。

集成工具Excel
查看安装方法

适用场景

  • Removing symbols and numbers from Chinese text columns
  • Preparing large Excel datasets for analysis
  • Combining sheets and exporting cleaned data

包含的文件(1)

  • SKILL.md2.2 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

Invalid_Data_Cleaning

This sub-skill covers one capability of the Excel workflow. For reading/counting/Parquet optimization, see the parent workflow SKILL.md.

Skill Steps

Step1 根据总行数判断是否数据量过大,若满足条件,则将 Excel 文件转换为 Parquet 格式提升读写效率,再读取数据进行后续分析。

import pandas as pd

file_path = "input_data.xlsx"
parquet_path = "temp_data.parquet"

# 读取 Excel 文件并转换为 Parquet 格式
xls = pd.ExcelFile(file_path)
dfs = []
for sheet in xls.sheet_names:
    df_sheet = pd.read_excel(xls, sheet_name=sheet)
    dfs.append(df_sheet)

# 合并所有 sheet 数据并写入 Parquet 文件
if dfs:
    df_all = pd.concat(dfs, ignore_index=True)
    df_all.to_parquet(parquet_path, engine='pyarrow', index=False)

# 读取 Parquet 文件用于后续处理
df = pd.read_parquet(parquet_path)

Step2 对目标文本字段中的特殊字符(如 #、-、数字)进行清洗,使用正则表达式仅保留中文字符。

import pandas as pd
import re

target_col = 'target_column' # 替换为实际需要清洗的列名

# 定义清洗函数
def clean_chinese_text(text):
    if pd.isna(text):
        return text
    s = str(text)
    # 提取所有中文字符(Unicode 范围:[一-鿿])
    chinese_chars = re.findall(r'[一-鿿]', s)
    cleaned = ''.join(chinese_chars)
    return cleaned if cleaned else ''

# 应用清洗函数
if target_col in df.columns:
    df[target_col] = df[target_col].apply(clean_chinese_text)

Step3 将清洗后的数据保存为表格文件(.xlsx),并在报告中提供本地下载链接。

import pandas as pd

# 保存清洗后的数据为 .xlsx 文件
output_path = "cleaned_data.xlsx"
df.to_excel(output_path, index=False)

print("清洗后的数据已保存至:", output_path)
# 生成本地文件下载链接
print("下载链接:", f"file://{output_path}")

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Compare counts and shares for two categories in Excel data. The skill cleans the selected columns, calculates differences and percentages, and produces a summary spreadsheet plus bar and pie charts.

中文简介正在准备中。当前显示英语简介。

  • Comparing counts for two categories
  • Visualizing category shares
  • Cleaning merged-cell data before counting
OpenSenseNova/SenseNova-Skills58482026年10月9日 更新

Analyze Excel data using custom categories, count distributions, and compare groups with charts. Score text using its length, technical terms, numbers, and selected complex concepts, then visualize difficulty levels and the relationship between text length and scores.

中文简介正在准备中。当前显示英语简介。

  • Summarize Excel records using custom categories
  • Compare category distributions across groups
  • Score and grade text using adjustable rules
OpenSenseNova/SenseNova-Skills58482026年10月9日 更新

Count categories in a chosen data column after removing missing values and placeholder labels. Calculate each category’s share and create high-resolution charts combining counts, percentages, pie charts, and horizontal bars for distribution reports.

中文简介正在准备中。当前显示英语简介。

  • Compare record counts across categories
  • Show category shares with pie and bar charts
  • Exclude placeholder labels before counting
OpenSenseNova/SenseNova-Skills58482026年10月9日 更新

Clean category and numeric fields in structured data, compare groups with counts and cross-tabulations, and produce charts, Excel results, and a Markdown report. Export visuals as high-resolution PNG and SVG files for use in reports.

中文简介正在准备中。当前显示英语简介。

  • Compare category distributions across groups
  • Clean category labels and fill merged-cell gaps
  • Export analysis results to Excel
OpenSenseNova/SenseNova-Skills58482026年10月9日 更新

Filter Excel data using candidate column names, RGB color values, text prefixes, and regular expressions, which match text patterns. Calculate totals and averages, then export results to Excel with highlighted rows and a download link.

中文简介正在准备中。当前显示英语简介。

  • Filtering rows by RGB values or identifier prefixes
  • Converting text to numbers and calculating totals or averages
  • Extracting matching commands with their identifiers and descriptions
OpenSenseNova/SenseNova-Skills58482026年10月9日 更新

Filter rows from Excel or previously converted Parquet files, a format designed for efficient data storage. Summarize matching records by group with counts, percentages, and totals, then export the summary to Excel.

中文简介正在准备中。当前显示英语简介。

  • Count records containing a specified risk level by work category
  • Calculate group shares among matching records
  • Summarize large datasets already converted to Parquet
OpenSenseNova/SenseNova-Skills58482026年10月9日 更新

查看 OpenSenseNova 的全部技能

报告此技能的问题