Author: Sergio Henrique
ARIMA and Online Learning in Financial Forecasting
I discuss the development of an online learning system using the Jane Street Real-Time Market Data Forecasting challenge as a practice ground for time-series forecasting. The project involves predicting the responder_6 variable using an ARIMA model, with a focus on adapting to new data by re-training the model whenever a new date_id is encountered. This approach leverages multiprocessing to meet strict time constraints
Walk Forward Validation on Jane Street Real-Time Market Data Forecast
Walk Forward Validation (WFV) involves a training window that moves forward in time, training the model on historical data and then validating it on future, unseen data points. Unlike traditional cross-validation where data is randomly split, WFV respects the sequence of time, making it ideal for datasets with time-dependent features like stock prices, weather patterns, or sales figures.
How to create and save charts with CrewAI agents and AWS S3
In the GDSC7 challenge, we’ve upgraded our agent system to create and display charts in response to user queries, using AWS S3 for image storage. The new chart.py tool leverages Pandas, Matplotlib, and Seaborn to generate various chart types, enhancing our system’s capability to present data visually. This integration allows us to effectively showcase complex information, such as the correlation between GDP and reading skills from the PIRLS 2021 study, improving user engagement and interaction.
Adding site and video as sources for CrewAI agent system
In my recent work on the GDSC7 challenge, I’ve been exploring how to enhance responses to subjective questions using data from the PIRLS 2021 study. By integrating the Embedchain package, I can efficiently connect to various sources like websites and videos, extract information, and store it in a vector database.
Enhancing Relational Database Agents with Retrieval Augmented Generation (RAG)
In the GDSC7 challenge, I tackled the issue of reducing API costs while preserving answer quality in agent systems. The solution involved implementing semantic search to efficiently handle gender-related queries without explicit gender columns in the database. By using embeddings for all string columns and combining questions with answers, we minimized unnecessary database queries, enhancing system efficiency and reducing costs.
How to override a method of instantiated object in python
In this post, I describe how I overcame AWS login challenges in a coding competition by using a method override trick. By defining a new function for authentication and dynamically replacing the existing method in an instantiated object, I was able to experiment with the Embedchain package without altering its class definition. This technique allowed for seamless integration with AWS services and added a valuable tool to my programming arsenal.