Table of Contents

Introduction
Setting Up Data Processing: Google Colab Environment Setup
Loading and Preprocessing CSV Files
Connecting to Neon and Importing CSV Data
Introduction to Supabase and Future Plans
Conclusion

1. Introduction

I am currently studying to become a data analyst.
To learn the basics of data preprocessing, I decided to leverage Google Colab and other tools.
This article outlines my progress, starting from data preprocessing with Google Colab to importing data into Neon and planning to use Supabase for future work.

2. Setting Up Data Processing: Google Colab Environment Setup

I used Google Colab to execute Python code.
I installed the necessary libraries, including:

pandas for data manipulation,
chardet for encoding detection,
glob for listing CSV files.

3. Loading and Preprocessing CSV Files

I uploaded CSV files from my local machine to Google Colab and started preprocessing:

Detected the file encoding using chardet.
Loaded the data into a pandas DataFrame.
Checked the shape of the data, verified missing values, and examined data types.
Summarized statistical information and reviewed the correlation matrix.

4. Connecting to Neon and Importing CSV Data

I successfully connected to Neon and created a table to store my preprocessed CSV data. The data import initially went well, but when I attempted to upload seven CSV files at once, the database size quickly exceeded three times the free plan’s limit. As a result, even after attempting to free up space, the capacity issue persisted.

In hindsight, I should have checked the storage limits more carefully in advance, and I could have reached out to Neon’s support team for guidance. However, since my priority was to continue studying data analysis without interruptions, I decided to close my account and switch to another platform. This decision was not due to any issue with Neon itself but rather my own lack of preparation.

That said, I still see great value in using Neon alongside Google Colab for integrating Python and SQL workflows. In the near future, I plan to revisit this combination, ensuring I better manage storage limits to fully leverage Neon’s capabilities for data processing.

5. Introduction to Supabase and Future Plans

I plan to continue my work with Supabase, as it offers an easier and more efficient platform for managing data and performing queries.
This tool will be part of my journey to streamline the data processing workflow.

6. Conclusion

In this article, I covered the steps I took to preprocess data using Google Colab, import it into Neon, and my decision to explore Supabase for future projects.
This journey marks an important step in my path to becoming a skilled data analyst.