As an engineer interested in the power grid, the goal of this evolving project is to explore available data, get feedback and learn from folks who are dealing with these problems in their day-to-day work. It's also an opportunity to learn and build a simple web app with common tools like React and Flask. The project contains three GitHub repositories:
Data exploration notebook | view on GitHub
The data exploration is inspired by Saul Griffith's flexible load profile discussion in Electrify, where home energy resources "take turns sucking up sunshine" to maximize use of abundant midday solar power (p. 89). According to Project Drawdown, grid flexibility is "critical to mainstreaming renewables, which are more variable than conventional energy sources." I looked at energy prices from Real-Time and Day Ahead markets, energy output data from NYISO (grid operator in New York) and ran simple algorithms on it to see how demand-side resources could be managed accordingly - one of the approaches that can help improve grid flexibility.
An important note/lesson is that Real-Time and Day Ahead market prices are not the prices the residential consumer pays. According to conEdison, the Day Ahead "hourly pricing reflects the wholesale rate per kWh. The information shows the extent to which wholesale electricity is more expensive during certain hours of the day" and per Resources for the Future's US Electricity Markets 101, "load-serving entities (the demand side) bid for that electricity to meet their customers’ energy demand."
Flask REST API | view on GitHub
I made the "Resources Scheduled Day Ahead" chart, from the data exploration above, interactive with a simple full-stack web app. The backend enables managing energy resources and provides optimal windows based on Day Ahead market prices.
Day Ahead market is based on "forecasted load for the next day," according to US Electricity Markets 101 - here I look at moving energy resources to lower priced parts of day, away from periods of high load. Though aggregators are outside the scope of this project, we could similarly explore maximum windows (representing high demand) for potentially releasing private solar or battery power back to the grid via aggregators in some utility programs.
Project uses Flask for REST APIs, queries Day Ahead prices from NYISO via GridStatus and finds the lowest price window for each resource. It also retrieves the 5 best windows for a selected resource and shows how they’re distributed throughout the day.
The light-weight backend implements the following fundamental elements:
Demo:
Data exploration notebook | view on GitHub
The data exploration is inspired by Saul Griffith's flexible load profile discussion in Electrify, where home energy resources "take turns sucking up sunshine" to maximize use of abundant midday solar power (p. 89). According to Project Drawdown, grid flexibility is "critical to mainstreaming renewables, which are more variable than conventional energy sources." I looked at energy prices from Real-Time and Day Ahead markets, energy output data from NYISO (grid operator in New York) and ran simple algorithms on it to see how demand-side resources could be managed accordingly - one of the approaches that can help improve grid flexibility.
An important note/lesson is that Real-Time and Day Ahead market prices are not the prices the residential consumer pays. According to conEdison, the Day Ahead "hourly pricing reflects the wholesale rate per kWh. The information shows the extent to which wholesale electricity is more expensive during certain hours of the day" and per Resources for the Future's US Electricity Markets 101, "load-serving entities (the demand side) bid for that electricity to meet their customers’ energy demand."
Flask REST API | view on GitHub
I made the "Resources Scheduled Day Ahead" chart, from the data exploration above, interactive with a simple full-stack web app. The backend enables managing energy resources and provides optimal windows based on Day Ahead market prices.
Day Ahead market is based on "forecasted load for the next day," according to US Electricity Markets 101 - here I look at moving energy resources to lower priced parts of day, away from periods of high load. Though aggregators are outside the scope of this project, we could similarly explore maximum windows (representing high demand) for potentially releasing private solar or battery power back to the grid via aggregators in some utility programs.
Project uses Flask for REST APIs, queries Day Ahead prices from NYISO via GridStatus and finds the lowest price window for each resource. It also retrieves the 5 best windows for a selected resource and shows how they’re distributed throughout the day.
The light-weight backend implements the following fundamental elements:
- Service-layer data caching
- Fall-back mechanism for retrieving today’s prices if tomorrow’s are not yet available
- ORM (SQLAlchemy) for reading and writing from local (SQLite) database
- Server-side validation
- Unit tests, including external data source mocking
Demo:
React UI | view on GitHub
The React UI is bootstrapped with the React+Vite template and makes requests to the above API. It displays data with Plotly, provides form validations and general error-handling for communicating with the backend.
Lessons Learned
The React UI is bootstrapped with the React+Vite template and makes requests to the above API. It displays data with Plotly, provides form validations and general error-handling for communicating with the backend.
Lessons Learned
- Biggest lesson is that the energy landscape is complicated, the contexts of the different prices in deregulated markets was interesting to learn about (linking US Electricity Markets 101 again here). The best way to appreciate that for me has been to dive into the data and evaluate my assumptions
- Flask and SQLAlchemy were relatively easy to get up and running, definitely plan on using them again
- Loved React's object-oriented way of organizing UI code, also plan on using it again
- The project has helped expand my mental model of the relationships between power producers, utilities and consumers and I'm pretty excited to continue learning
- Time-of-Use pricing does potentially allow for consumers' potential shifting of energy use to off-peak windows - need to further explore how that data could be integrated to reflect any consumer savings
- Resources likely need to prefer certain times of day, and provide option for "depending on" another resource, i.e. only run dryer when washing machine has already run
- In the web-app, there is no concept of a user currently (i.e. different list of resources for different users)
- The grid price query can be further customized with additional regions and grid operators
- Application is not currently deployed publicly
- Please see additional limitations within the data exploration notebook
- Griffith, S. (2022). Electrify: An Optimist’s Playbook for Our Clean Energy Future. MIT Press.
- GridStatus
- US Electricity Markets 101
- Grid Flexibility on Project Drawdown
- NYISO Dashboard
- React + Vite template
- React Plotly.js