An interactive data dashboard for exploring foreign-born population demographics across Massachusetts' 26 Gateway Cities using American Community Survey (ACS) 5-year estimates (2012–2024). Built for journalists and researchers to analyze immigration trends, economic assimilation, and community demographics.
- Overview — Statewide summary of foreign-born population shares across all MA places
- Per Capita Comparison — Side-by-side metric comparisons across cities
- City Profile — Deep-dive into a single city's demographics, income, education, and housing
- Country of Origins — Granular country-by-country breakdown of foreign-born populations
- Map View — Interactive Leaflet map with markers sized/colored by foreign-born share
- Trends — Time-series charts (2012–2024) for any metric across cities
- AI Chatbot — Conversational interface powered by Google Gemini for natural-language queries
The chatbot combines deterministic data analysis with Gemini LLM rewriting. Supported question types:
| Question Type | Example |
|---|---|
| City profile with statewide comparison | "How has the foreign-born population changed in Lowell?" |
| Foreign-born population growth ranking | "Which Gateway Cities have had the greatest foreign-born growth?" |
| Granular country-of-origin breakdown | "What is the breakdown of foreign-born populations in Lawrence by country?" |
| Fastest-growing subgroups | "What are the fastest-growing foreign-born subgroups in Lowell?" |
| Economic assimilation indicators | "What are the economic indicators for foreign-born in Quincy?" |
| Economic integration ranking | "Which cities show the strongest economic integration over time?" |
| City comparison | "How do Gateway Cities compare to other MA cities and statewide averages?" |
| Trend analysis | "What is the foreign-born trend in Chelsea since 2010?" |
| Poverty / FB rankings | "Which Gateway Cities have the lowest poverty rates?" |
| Layer | Technology |
|---|---|
| Frontend | React 19, Vite 7, Recharts, Leaflet / React-Leaflet |
| Backend | Flask 3, Python 3.12 |
| Data | Pandas, PyArrow (Parquet files) |
| AI | Google Gemini API (gemini-2.5-flash) |
| Data Source | U.S. Census Bureau ACS 5-year estimates |
gateway_cities/
├── backend/
│ ├── app.py # Flask API routes (13 endpoints)
│ ├── rag_index/
│ │ └── index.json # RAG index for chatbot context
│ └── services/
│ ├── chat_service.py # Chatbot intent routing + Gemini integration
│ ├── data_store.py # Parquet data loaders for all datasets
│ └── rag.py # RAG retrieval utilities
├── data/
│ ├── raw/ # Original ACS CSV downloads
│ ├── interim/ # Year-by-year intermediate parquets (2012–2024)
│ └── processed/ # Final parquet files consumed by the API
│ ├── cities_master.parquet
│ ├── country_of_origin.parquet
│ ├── education.parquet
│ ├── employment_income.parquet
│ ├── foreign_born_core.parquet
│ ├── homeownership.parquet
│ ├── median_income.parquet
│ └── poverty_by_nativity.parquet
├── frontend/
│ ├── public/data/
│ │ └── gateway_cities.geojson # City boundary polygons
│ └── src/
│ ├── App.jsx # Root component, tab navigation
│ ├── api/cities.js # API client
│ └── components/
│ ├── ChatBot.jsx # AI chatbot interface
│ ├── CityProfile.jsx # Single-city deep-dive
│ ├── CountryOrigins.jsx # Country-of-origin breakdown
│ ├── MapView.jsx # Leaflet map visualization
│ ├── PerCapitaComparison.jsx # Cross-city metrics
│ ├── SearchableCitySelect.jsx # City selector dropdown
│ └── TrendsView.jsx # Time-series charts
├── scripts/
│ ├── fetch_acs_data.py # Fetch ACS data from Census API
│ ├── fetch_gateway_cities_geojson.py # Fetch city boundary GeoJSON
│ ├── 00_validate_raw.py # Step 1: Validate raw CSV downloads
│ ├── 10_normalize_places.py # Step 2: Normalize city/place names
│ └── 20_build_per_capita_metrics.py # Step 3: Build per-capita parquets
├── .env # Environment variables (API keys)
├── requirements.txt # Python dependencies
└── README.md
- Python 3.12+
- Node.js 18+
- A Google Gemini API key (for the chatbot)
git clone https://github.com/pratrt141098/gateway_cities.git
cd gateway_citiespython -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txtCreate a .env file in the project root:
FLASK_ENV=development
DATA_DIR=./data/processed
GEMINI_API_KEY=your_gemini_api_key_here
GEMINI_MODEL=gemini-2.5-flashcd frontend
npm installYou need two terminals open simultaneously.
Terminal 1 — Backend (Flask)
cd gateway_cities
source .venv/bin/activate
python backend/app.pyThe API runs at http://localhost:3000.
Terminal 2 — Frontend (Vite)
cd gateway_cities/frontend
npm run devThe dashboard runs at http://localhost:5173.
Open http://localhost:5173 in your browser.
| Endpoint | Method | Parameters | Description |
|---|---|---|---|
/api/health |
GET | — | Health check |
/api/cities |
GET | — | Master list of all cities |
/api/foreign-born |
GET | city, city_type |
Foreign-born population statistics |
/api/country-of-origin |
GET | city |
Country-of-origin breakdown |
/api/education |
GET | city |
Educational attainment |
/api/homeownership |
GET | city |
Homeownership rates |
/api/employment-income |
GET | city |
Employment and income data |
/api/poverty |
GET | city |
Poverty rates by nativity |
/api/median-income |
GET | city |
Median income by nativity |
/api/map-stats |
GET | — | Map visualization data |
/api/time-series |
GET | city, metric |
Historical data (2012–2024) |
/api/chat |
POST | {"message": "..."} |
AI chatbot |
Available time-series metrics: fb_pct, unemployment_rate, median_income, poverty_rate, bachelors_pct, homeownership_pct, fb_income
The processed parquet files are included in the repo. If you need to rebuild them from raw ACS data:
source .venv/bin/activate
python scripts/fetch_acs_data.pyThis downloads ACS for all MA places (2012–2024).
python scripts/00_validate_raw.py
python scripts/20_build_per_capita_metrics.pyscripts/10_normalize_places.py is a legacy single-year helper and is not part of the current multi-year pipeline.
| Table ID | Description |
|---|---|
| B05001 | Nativity in the United States |
| B05002 | Place of birth by citizenship status |
| B05003 | Nativity by place of birth |
| B05006 | Place of birth for foreign-born population |
| B05010 | Ratio of income to poverty level by nativity |
| B06011 | Median income by nativity |
| B15002 | Sex by educational attainment |
| B25003 | Tenure (owner vs renter) |
| DP03 | Selected economic characteristics |
| S0501 | Selected characteristics of the foreign-born population |
All dashboard metrics are built from U.S. Census Bureau ACS 5-year estimates, not ACS 1-year estimates. This keeps the methodology consistent across all Gateway Cities and comparison places, including smaller geographies where ACS 1-year data may not be available. Census Reporter often defaults to ACS 1-year for eligible places, so its values can differ from this dashboard.
Median household income is read directly from ACS Data Profile table DP03_0062E. It is not calculated from B19001; B19001 is an income distribution table. The comparable detailed-table median is generally B19013_001E.
| Dashboard field | Source table / variable | Calculation |
|---|---|---|
total_pop |
B05002_001E |
Total population estimate. |
foreign_born |
B05002_013E |
Foreign-born population estimate. |
fb_pct |
B05002_013E, B05002_001E |
foreign_born / total_pop * 100. |
fb_naturalized |
B05002_014E |
Foreign-born naturalized U.S. citizens. |
fb_naturalized_pct |
B05002_014E, B05002_013E |
fb_naturalized / foreign_born * 100. |
fb_not_citizen |
B05002_021E |
Foreign-born residents who are not U.S. citizens. |
fb_not_citizen_pct |
B05002_021E, B05002_013E |
fb_not_citizen / foreign_born * 100. |
country, estimate |
B05006_*E |
One row per country/place-of-birth variable. Region labels are derived with country_converter. |
total_25plus |
B15002_001E |
Population age 25 and older. |
hs_pct |
B15002_011E, B15002_028E, B15002_001E |
High school diploma counts for male and female divided by total age 25+. |
bachelors_pct |
B15002_015E, B15002_016E, B15002_017E, B15002_018E, B15002_032E, B15002_033E, B15002_034E, B15002_035E, B15002_001E |
Bachelor's degree or higher divided by total age 25+. |
bach_only_pct |
B15002_015E, B15002_032E, B15002_001E |
Bachelor's degree only divided by total age 25+. |
advanced_pct |
B15002_016E, B15002_017E, B15002_018E, B15002_033E, B15002_034E, B15002_035E, B15002_001E |
Advanced degree divided by total age 25+. |
total_housing_units |
B25003_001E |
Occupied housing units. |
owner_occupied |
B25003_002E |
Owner-occupied housing units. |
renter_occupied |
B25003_003E |
Renter-occupied housing units. |
homeownership_pct |
B25003_002E, B25003_001E |
owner_occupied / total_housing_units * 100. |
employed |
DP03_0004E |
Civilian labor force employed estimate. |
unemployed |
DP03_0005E |
Civilian labor force unemployed estimate. |
unemployment_rate |
DP03_0004E, DP03_0005E |
unemployed / (employed + unemployed) * 100. |
median_household_income |
DP03_0062E |
Median household income in the past 12 months, inflation-adjusted to the ACS release year. |
mean_household_income |
DP03_0063E |
Mean household income in the past 12 months, inflation-adjusted to the ACS release year. |
poverty_rate |
DP03_0119PE |
Percent of all people whose income in the past 12 months is below the poverty level. |
median_income_total |
B06011_001E |
Median income in the past 12 months for the total population in the table universe. |
median_income_foreign_born |
B06011_005E |
Median income in the past 12 months for foreign-born residents. |
fb_poverty_universe |
B05010_002E |
Foreign-born poverty-status universe. |
fb_below_poverty |
B05010_003E |
Foreign-born population below poverty level. |
fb_poverty_pct |
B05010_003E, B05010_002E |
fb_below_poverty / fb_poverty_universe * 100. |
The 26 Massachusetts Gateway Cities tracked in this dashboard:
Attleboro, Barnstable, Brockton, Chelsea, Chicopee, Everett, Fall River, Fitchburg, Haverhill, Holyoke, Lawrence, Leominster, Lowell, Lynn, Malden, Methuen, New Bedford, Peabody, Pittsfield, Quincy, Revere, Salem, Springfield, Taunton, Westfield, Worcester
All other Massachusetts places are labeled as Other Cities in MA for comparison.
- Geographic scope: 260 Massachusetts places (cities, towns, CDPs)
- Time range: 2012–2024
- Metrics: Foreign-born share, median income (overall & by nativity), unemployment, poverty, educational attainment, homeownership
- Country of origin: 150+ specific countries, grouped by region
This project uses publicly available data from the U.S. Census Bureau American Community Survey.