Changes in TOC with first suggestion

[ronnyhdez]

Ref #427 based on this comment: #427 (comment)

[ateucher]

/deploy-preview

[github-actions]

🚀 Deployed on https://deploy-preview--openscapes-org-preview.netlify.app

[alexishunzinger]

I like the descriptions of How-To and Tutorial in parentheses, great way to briefly describe those.

I'm hung up on the "chapters", Examples by Data Providers and External Tutorials

1. Examples by Data Providers
   It is new (this week) guidance on the Earthdata website to minimize the use of DAAC names. As far as users are concerned, all data is in the cloud archive, centrally accessed from the Earthdata website once all DAAC websites are merged. Plus, in a year's time DAACs will restructure into thematic teams (SETs).

While the phasing out of DAACs and DAAC branding will take a while and is currently inconsistent on the Earthdata website, we can stay ahead of the change and maybe model a better way to organize these in the cookbook.

2. External Tutorials
   I clicked around the tutorials in this section to see what was here. It appears nothing is really "external" to NASA Earthdata, DAACs or Openscapes, most are from Hackdays/Hackathons/Workshops. I assumed external meant, for example, directly from Xarray's website.

Same sentiment here as in 1. -- perhaps the chapters could change to reflect content of these tutorials, rather than source.

All of this said with no solution in mind (whoops!), though I'd be happy to spend more time thinking about it and discussing with others. I also could be okay with these chapters if we would like to get a reorg done sooner than later. There are just a lot of changes coming that we do not yet know what they look like in practice.

[danielfromearth]

@ronnyhdez, @alexishunzinger

Some additional thoughts:

1. Examples

1. Examples by Data Providers
   It is new (this week) guidance on the Earthdata website to minimize the use of DAAC names. As far as users are concerned, all data is in the cloud archive, centrally accessed from the Earthdata website once all DAAC websites are merged. Plus, in a year's time DAACs will restructure into thematic teams (SETs).

While the phasing out of DAACs and DAAC branding will take a while and is currently inconsistent on the Earthdata website, we can stay ahead of the change and maybe model a better way to organize these in the cookbook.

How about organizing examples by 'sphere' and access instead of DAACs? We could put the 'sphere' as the Table of Contents section heading instead of DAAC, then have each notebook be titled with an access pattern first and relevant science data source in parentheses: such as "Harmony-py/Subsetting (ECOSTRESS dataset)" or "Kerchunk Workflow (Sea Surface Height).

2. External Tutorials

2. External Tutorials
   I clicked around the tutorials in this section to see what was here. It appears nothing is really "external" to NASA Earthdata, DAACs or Openscapes, most are from Hackdays/Hackathons/Workshops. I assumed external meant, for example, directly from Xarray's website.

Same sentiment here as in 1. -- perhaps the chapters could change to reflect content of these tutorials, rather than source.

All of this said with no solution in mind (whoops!), though I'd be happy to spend more time thinking about it and discussing with others. I also could be okay with these chapters if we would like to get a reorg done sooner than later. There are just a lot of changes coming that we do not yet know what they look like in practice.

What about renaming this "External Tutorials" section to something like "Archived"; then inspecting the contents to see if there is material that should be sortof 'promoted' to be in the main "tutorials" or "how-tos" sections. It seems to me like archiving is the main reason for this – to be able to reference previous material. If there's anything that is important for users to learn from currently, then it deserves to be made into a first-class tutorial/how-to in those sections?

3. Repo files

Are we going to move the files themselves in the repo as part of this PR too?

[danielfromearth]

Perhaps the "When To Cloud" and "Glossary & Cheatsheets" sections would be better placed in the "Reference" section?

Suggested change

	- text: "When To Cloud"
	href: when-to-cloud.qmd
	- text: "Cloud Environment Setup"
	href: environment-setup/index.qmd
	- text: "Glossary & Cheatsheets"
	href: glossary.qmd
	- text: "Cloud Environment Setup"
	href: environment-setup/index.qmd

[ronnyhdez]

I agree!

[ateucher]

I agree about Glossary and Cheat sheets moving to "References", but I think @alexishunzinger current WIP cheat sheet should be given higher exposure when she's ready to share it. People really like cheat sheets, and would be less likely to find it if it's in "References" I think.

I think there's an argument for "When to Cloud" to still live in "Getting Started"

[danielfromearth]

I think I'm okay with not having an "In Development" section. I suppose that means we'll keep in-development work to branches other than main?

[ronnyhdez]

Yes, I think it would be better to use branches, instead of keeping the files floating in the main branch. And using releases after reviews.

I removed that section from the TOC, but yes, we need discuss further how to handle those files. I feel that some of the files could be better tracked as issues (task/to-do). For content that is further developed, but still not ready to be part of the published version of the book, I suggest moving it to its intended location and opening a draft PR for review in different branches as you said.

[ateucher]

I agree with this

[alexishunzinger]

@ronnyhdez @danielfromearth Thanks for the concrete suggestions Danny!

How about organizing examples by 'sphere' and access instead of DAACs? We could put the 'sphere' as the Table of Contents section heading instead of DAAC, then have each notebook be titled with an access pattern first and relevant science data source in parentheses: such as "Harmony-py/Subsetting (ECOSTRESS dataset)" or "Kerchunk Workflow (Sea Surface Height).

The five spheres we anticipate Earthdata sorting us into are

• Atmosphere
• Biosphere
• Cryosphere
• Geosphere
• Hydrosphere

I like the structure Access Pattern (Science Data Source) to start with. This is also leading me back to the data access cheat sheet as perhaps a map for us to use to make sure we cover all these methods in our cookbook tutorials.

What about renaming this "External Tutorials" section to something like "Archived"; then inspecting the contents to see if there is material that should be sortof 'promoted' to be in the main "tutorials" or "how-tos" sections. It seems to me like archiving is the main reason for this – to be able to reference previous material. If there's anything that is important for users to learn from currently, then it deserves to be made into a first-class tutorial/how-to in those sections?

Agree with this method. Outside of this initial cookbook reorganization, we have content to review and decide what to promote.

[jules32]

Great ideas – and @alexishunzinger I love the idea of your cheatsheet being the map!

[ronnyhdez]

Hi @alexishunzinger and @danielfromearth!

Thanks for your input! That helps me to understand better what can be done!

I was thinking on grouping those tutorials in external because there are several files under a external folder in the repo. Not all of them are listed in the _quarto.yml file. This means that we have files in the repo that are not shown in the book. This is how the external folder contents look like:

external/
├── appeears_csv_cloud_access.ipynb
├── cof-zarr-reformat.ipynb
├── data_access_direct_S3.ipynb
├── data_discovery_cmr.ipynb
├── data_discovery_cmr-stac_api.ipynb
├── harmony_subsetting.ipynb
├── nasa_earthdata_authentication.ipynb
├── on-prem_cloud.ipynb
├── sentinel-6_opendap_access_gridding.ipynb
├── xarray.ipynb
└── zarr-eosdis-store.ipynb

0 directories, 11 files

From those files, the book is currently using 3:

appeears_csv_cloud_access.ipynb
cof-zarr-reformat.ipynb
zarr-eosdis-store.ipynb

We have some content to review. I can start moving some of the files, offer those changes in PRs (being careful of not loosing the git history info), and from there, we can review a PR and accept the proposed change or decide what action to do with the file.

We have also the In development section with files to review. Most of them are ideas that could be documented in issues rather than a file.

I can open a new issue to address "organizing examples by 'sphere' and access". So we can document there what needs to be done and other details.

Regarding moving files in this PR: my suggestion would be to keep this PR focused on defining the high-level structure, with only minimal file relocation. Once we agree on that structure, we could open a follow-up PR (or several) to move files and document the review process and decisions there. Happy to hear what others think or if there’s a better approach!

[ateucher]

I love where this is going!

1. Examples

I agree about removing DAAC names. Is there a reason why these wouldn't just go into "tutorials" or "how-tos"? If I was coming to the cookbook with no prior experience, I wouldn't know the difference between "examples" and "tutorials".

What do you think about keeping the "how-tos" structure as-is, but structure the "tutorials" by sphere, and putting the notebooks currently in "Examples" into tutorials by sphere (using the Access Pattern (Science Data Source) naming pattern? In my head it feels like most tutorials would be science-focused and thus fit in a sphere (or more than one), and how-tos would be mostly tool/service focused

2. External

I agree about flagging and auditing these in some way. We could just leave them all in "External" (and expose them as @ronnyhdez has done here) to audit and place in the appropriate place as a discrete task (or set of discrete tasks), with the ultimate goal of removing this directory and having the relevant ones in the appropriate how-to or tutorial section.

3. Reference

A lot of the "Alternate Access Methods" look like how-tos to me. Can they be moved there?

[ateucher]

I like the organization overall, but wonder if Welcome should be on its at the top, not in a folder?

[ateucher]

However, if "When to Cloud" and "Glossary and Cheatsheets" are moved to the "Reference Section, that would leave only "Cloud Environment Setup" in the getting started... section.

[ateucher]

I think we can remove all of this since it's moved to https://openscapes.cloud

[ateucher]

I propose we move the jupyterhub "Getting Started" to https://openscapes.cloud and update it. It's less about accessing/using cloud data, and getting set up in our specific jupyterhub. That said, I think it's worth linking to resources like this from the cookbook

[ateucher]

I agree about Glossary and Cheat sheets moving to "References", but I think @alexishunzinger current WIP cheat sheet should be given higher exposure when she's ready to share it. People really like cheat sheets, and would be less likely to find it if it's in "References" I think.

I think there's an argument for "When to Cloud" to still live in "Getting Started"

[ateucher]

I agree with this

[asteiker]

I'm trying to catch up here so I may not be incorporating all of the prior feedback, though hopefully there's some overlap:

Perhaps the greatest change here is the move of "Our Cookbooks" to the bottom under "Community and Resources". Given our tutorial visioning, I think we want to prominently describe the broader cookbook landscape (hub and spoke) and provide users with a mental model of how these resources interconnect.

In fact, I'm wondering if we ought to reference our existing DAAC cookbooks more prominently in each of the SET categories as I think is being discussed above?

One other note for now (I will also go through the .yml file in more detail):

For "External Tutorials":
Many of these look to be pulled from our prior Hackathon content so I'm not sure these should be categorized as "external" per se. But again I think @ateucher may also be suggesting that these are moved or recategorized.

[danielfromearth]

Great discussion during the Cookbook Coworking today!

Perhaps the greatest change here is the move of "Our Cookbooks" to the bottom under "Community and Resources". Given our tutorial visioning, I think we want to prominently describe the broader cookbook landscape (hub and spoke) and provide users with a mental model of how these resources interconnect.

I agree, @asteiker, that the broader cookbook landscape / hub-and-spoke vision deserves more prominence than just a list of links in a "Our Cookbooks" page. Seems to me like it is underselling and being less helpful for users to understand/navigate the hub-and-spoke vision and relationships between cookbooks. Could be a separate, focused discussion — so, this PR focus on the high-level structure, and a new issue/PR focus on highlighting/improving the cross-Cookbook vision?

[ateucher]

Great questions/observations @asteiker! Yup, we all agreed that we need to give the whole cookbook landscape more profile and visibility. I think the agreement for now was to make this PR really about the high-level structure, knowing that a key next step is to profile the Cookbook landscape properly.

As for the external/ tutorials, the plan for now is to move them into the how-tos/ and tutorials/ and not separate them out. Ditto for the "Examples by Data Provider". Once the top-level structure is in place and we see what notebooks we have where, we can move on to how to subcategorize them, e.g., by SET, by Access Pattern, by mission, etc...

[ronnyhdez]

Changes are implemented in PR #444 built on top of this commits.