150 european higher education sector observatory

[Schwarzbleistift]

closes #150

[doug-leasure]

Thanks, @Schwarzbleistift! We have a few steps remaining to add these new data into the database and dashboard.

Everything is outlined in our documentation for writing custom data into the database: https://docs.mapineq.org/database_management#import-custom-data

1. @Schwarzbleistift, could you please put the final csv files into our network drive at: K:/DemSci/projects/2024_Mapineq/final_data/european_higher_ed_observatory/?
2. @dhvalden , are you available to help us write this data into the database? We can either write directly into the production database, or we could liase with Geodienst to write the data onto the dev server first.
3. @Schwarzbleistift , please provide the following metadata to include with the data in the database:
   • URL to the home page of the project where the data originate
   • URL to the metadata for the specific dataset that we are including in our database (prefer machine-readable metadata, if available)
   • URL to download the data directly (prefer a permanent DOI, if available)
   • URL to the data license associated with the dataset (prefer full license text e.g. CC-BY 4.0 or comparable, if available).

[Schwarzbleistift]

Thanks @doug-leasure. I have put the data into the desginated folder. And following are the links required:

1. The home page: https://national-policies.eacea.ec.europa.eu/eheso
2. The metadata: https://national-policies.eacea.ec.europa.eu/eheso/methodology-and-technical-documentation
3. Link to download the data: https://eter-project.com/data/data-for-download-and-visualisations/database/
4. License: https://eter-project.com/about/using-eter-data/

[doug-leasure]

Hi @Schwarzbleistift . I am going to make notes here of the changes that I requested during our meeting yesterday so that @dhvalden is aware.

We discussed how it would be better to combine the csv's as much as possible to minimise the number of datasets that we are ultimately adding into the dashboard. We identified examples where we can combine data by including multiple filters in a single dataset (e.g. gender, field, indicator).

When this has been done, we will ask @dhvalden to load the tables into the database/dashboard.

[Schwarzbleistift]

Hi @Schwarzbleistift . I am going to make notes here of the changes that I requested during our meeting yesterday so that @dhvalden is aware.

We discussed how it would be better to combine the csv's as much as possible to minimise the number of datasets that we are ultimately adding into the dashboard. We identified examples where we can combine data by including multiple filters in a single dataset (e.g. gender, field, indicator).

When this has been done, we will ask @dhvalden to load the tables into the database/dashboard.

hi @doug-leasure. Yes, I modified the script and submitted a commit to combine some of the csv. Also, I updated the csv after combination into the k drive: K:/DemSci/projects/2024_Mapineq/final_data/european_higher_ed_observatory/

[dhvalden]

so, as discussed in the meeting, I''ll request some minor adjustments ot the data to streamline the publication uploading process:

• The geo_source column must be at the end of the table and and the end of every csv in order for the dashboard to skip it.
• A column id must be added as the fist column in every csv. It can be empty or filled with sequential numbers from 0 onwards. We will use this columns as KEY for the table, so make sure to have only unique values here.
• As Doug mentioned the column indicator seems to be a constant. So, we should delete it from the csv. Make sure that is a constant in all cases tho. As a general rule, columns that don't provide information for the filters (like constants or all unique values) should be dropped.
• Create a new option for gender called gender_ratio which will be populated with the gender ratio.
• Create a filter option for head count (HC) and full time (FT) in the gender column.
• Finally, some minor comments on file name convention: Lets use snake_case to name the files, meaning that we will use all lowercase and underscore instead of spaces.

I will tag @Schwarzbleistift for implementation of this steps and @doug-leasure to see you agree. @Schwarzbleistift any questions you have or if you wish to work together on this let me know :)

[dhvalden]

data added to database and scripts committed in 5baca2d

[dhvalden]

• New column for FT of HC
• all is gender_ratio
• why is the map not showing correctly?

[Schwarzbleistift]

• New column for FT of HC
• all is gender_ratio
• why is the map not showing correctly?

Hi @dhvalden I think the data are now ready to go. Can you please upload the data without displaying them temporily, as we are still waiting for the authorization from the data provider?