bucky.util package¶
Submodules¶
bucky.util.read_config module¶
bucky.util.readable_col_names module¶
bucky.util.update_data_repos module¶
-
bucky.util.update_data_repos.
distribute_data_by_population
(total_df, dist_vect, data_to_dist, replace)¶ Distributes data by population across a state or territory.
- Parameters
total_df (Pandas DataFrame) – DataFrame containing confirmed and death data indexed by date and FIPS code
dist_vect (Pandas DataFrame) – Population data for each county as proportion of total state population, indexed by FIPS code
data_to_dist (Pandas DataFrame) – Data to distribute, indexed by data
replace (boolean) – If true, distributed values overwrite current historical data in DataFrame. If false, distributed values are added to current data
- Returns
total_df – Modified input dataframe with distributed data
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
distribute_mdoc
(df, csse_deaths_file)¶ Distributes Michigan Department of Corrections data across Michigan counties by population.
- Parameters
df (Pandas DataFrame) – Current historical DataFrame indexed by FIPS and date, which includes MDOC and FCI data
csse_deaths_file (string) – File location of CSSE deaths file (contains population data)
- Returns
df – Modified historical dataframe with Michigan prison data distributed and added to Michigan data
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
distribute_nyc_data
(df)¶ Distributes NYC case data across the six NYC counties.
- Parameters
df (Pandas DataFrame) – DataFrame containing historical data indexed by FIPS and date
add deprecation warning b/c csse has fixed this (TODO) –
- Returns
df – Modified DataFrame containing corrected NYC historical data indexed by FIPS and date
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
distribute_territory_data
(df, add_american_samoa)¶ Distributes territory-wide case and death data for territories.
Uses county population to distribute cases for US Virgin Islands, Guam, and CNMI. Optionally adds a single case to the most populous American Samoan county.
- Parameters
df (Pandas DataFrame) – Current historical DataFrame indexed by FIPS and date, which includes territory-wide case and death data
add_american_samoa (boolean) – If true, adds 1 case to American Samoa
- Returns
df – Modified historical dataframe with territory-wide data distributed to counties
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
distribute_unallocated_csse
(confirmed_file, deaths_file, hist_df)¶ Distributes unallocated historical case and deaths data from CSSE.
JHU CSSE data contains state-level unallocated data, indicated with “Unassigned” or “Out of” for each state. This function distributes these unallocated cases based on the proportion of cases in each county relative to the state.
- Parameters
confirmed_file (string) – filename of CSSE confirmed data
deaths_file (string) – filename of CSSE death data
hist_df (Pandas DataFrame) – current historical DataFrame containing confirmed and death data indexed by date and FIPS code
- Returns
hist_df – modified historical DataFrame with cases and deaths distributed
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
get_county_population_data
(csse_deaths_file, county_fips)¶ Uses JHU CSSE deaths file to get county-level population data as as fraction of total population across requested list of counties.
- Parameters
csse_deaths_file (string) – filename of CSSE deaths file
county_fips (array-like) – list of FIPS to return population data for
- Returns
population_df – DataFrame with population fraction data indexed by FIPS
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
get_timeseries_data
(col_name, filename, fips_key='FIPS', is_csse=True)¶ Takes a historical data file and reduces it to a dataframe with FIPs, date, and case or death data.
- Parameters
col_name (string) – Column name to extract from data.
filename (string) – Location of filename to read.
fips_key (string, optional) – Key used in file for indicating county-level field.
is_csse (boolean) – Indicates whether the file is CSSE data. If True, certain areas without FIPS are included.
- Returns
df – Dataframe with the historical data indexed by FIPS, date
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
git_pull
(abs_path)¶ Updates a git repository given its path.
- Parameters
abs_path (string) – Abs path location of repository to update
-
bucky.util.update_data_repos.
process_csse_data
()¶ Performs pre-processing on CSSE data.
CSSE data is separated into two different files: confirmed cases and deaths. These two files are combined into one dataframe, indexed by FIPS and date with two columns, Confirmed and Deaths. This function distributes CSSE that is either unallocated or territory-wide instead of county-wide. Michigan data from the state Department of Corrections and Federal Correctional Institution is distributed to Michigan counties. New York City data which is currently all placed in one county (New York County) is distributed to the other NYC counties. Territory data for Guam, CNMI, and US Virgin Islands is also distributed. This data is written to a CSV.
-
bucky.util.update_data_repos.
process_usafacts
(case_file, deaths_file)¶ Performs preprocessing on USA Facts data.
USAFacts contains unallocated cases and deaths for each state. These are allocated across states based on case distribution in the state.
- Parameters
case_file (string) – Location of USAFacts case file
deaths_file (string) – Location of USAFacts death file
- Returns
combined_df – USAFacts data containing cases and deaths indexed by FIPS and date.
- Return type
Pandas DataFrame
-
bucky.util.update_data_repos.
update_covid_tracking_data
()¶ Downloads and processes data from the Atlantic’s COVID Tracking project to match the format of other preprocessed data sources.
The COVID Tracking project contains data at a state-level. Each state is given a random FIPS selected from all FIPS in that state. This is done to make aggregation easier for plotting later. Processed data is written to a CSV.
-
bucky.util.update_data_repos.
update_repos
()¶ Uses git to update public data repos.
-
bucky.util.update_data_repos.
update_usafacts_data
()¶ Retrieves updated historical data from USA Facts, preprocesses it, and writes to CSV.
bucky.util.util module¶
-
class
bucky.util.util.
TqdmLoggingHandler
(level=0)¶ Bases:
logging.Handler
-
emit
(record)¶ Do whatever it takes to actually log the specified logging record.
This version is intended to be implemented by subclasses and so raises a NotImplementedError.
-
-
bucky.util.util.
bin_age_csv
(filename, out_filename)¶
-
bucky.util.util.
cache_files
(*argv)¶
-
bucky.util.util.
date_to_t_int
(dates, start_date)¶
-
class
bucky.util.util.
dotdict
¶ Bases:
dict
dot.notation access to dictionary attributes
-
bucky.util.util.
estimate_IFR
(age)¶
-
bucky.util.util.
map_np_array
(a, d)¶
-
bucky.util.util.
unpack_cache
(cache_file)¶