Skip to content

A few suggestions for gr.Workflow #13665

Description

@abidlabs

Testing gr.Workflow a bit for image -> webpage, and noticed a few things that would help the UX/DX. I'm opening a PR to resolve some of them, but not sure if others are intentional choices, so wanted to create a standalone issue as well:

  • When launching a gr.Workflow, have the default behavior to be to open the Workflow in a new tab with the write token already passed in as a query parameter. This is the same behavior as jupyter notebooks and makes it easier to immediately get going with a Workflow. Part of Workflow UX improvements, and fix gradio skills add clobbering skills via symlinked dirs #13666
  • Autocreate input/output nodes for model nodes. This is already the case for Space nodes, but not for model nodes. Sometimes the input node does appear inside the model node. Not sure when/why that happens, but regardless, not having the output node can make it seem like the node runs but doesn't produce anything:
Image
  • For image output nodes, html output nodes, and probably for all other output nodes, it would be great if you could resize the node. Depending on what you're doing, it might be useful e.g. to be able to compare output images at a larger size. In addition, it would be great if you could view any individual input or output image in full screen mode.
  • I still have a hard time finding some Spaces. For example, how do I get the background removal Space? I clicked Image but couldn't find any Space for background removal (no explicit category, nor does anything show up when I type "background"). When I pasted the Space URL directly, it worked. But even then, this simple workflow errors out -- am I doing something wrong?
Image
  • Speaking of the bottom navbar, why do we have a button for Input, but not Output? Do we even need one for Input or is it sufficient that users can click on the input/output ports on any Space or Model node? Also nit, but since Data refers to HF Datasets IIUC, I think we should just call it Dataset instead of Data, which implies more general data sources imo.
  • I think the audio input should accept mic recording (often easier than finding an audio file) and image compnoent should similarly accept webcam
  • Have ctrl/cmd+z for undo and ctrl/cmd+y for unundo would be nice
  • Model nodes for vision-language models can't run: image-text-to-text is mapped to the visual_question_answering endpoint, but no Inference Provider serves that task for any model on the Hub (the router routes every model with that tag as conversational), so the node always fails with Task 'visual-question-answering' not supported for provider 'together'. The same is true of document-question-answering, image-to-text and audio-classification — 0 provider-served models each — though those only affect models that are themselves unserved. Fixed for image-text-to-text in Workflow UX improvements, and fix gradio skills add clobbering skills via symlinked dirs #13666
  • It would be great if we could copy-paste error messages for easy copy-pasting into llms
  • We should set streaming=True for inference providers, otherwise we end up with mysterious 504s (see my internal message), did that here: Workflow UX improvements, and fix gradio skills add clobbering skills via symlinked dirs #13666
  • I think you can't actually call Workflows that are hosted on Spaces via API because they are ouath-gated. Opening a fix for that here: Let API callers supply a token for gr.OAuthToken endpoints #13667. After this is in, we can update the docs for the Workflow view API to include the oauth_token parameter
  • So I created this workflow Space: https://huggingface.co/spaces/abidlabs/vlm-screenshot-to-webpage. But it would be really nice if I could provide some preopulated (cached) examples that a user immediately sees when they land on the Space. Similar to cached examples in standard Gradio apps.
  • I don't know if there's any way to fix this, but often times nodes that are not initially overlapping, will overlap once you populate their contents, e.g. if you look at the input nodes here:
Image

Sorry for all of these things, but wanted to get them down while my experience was fresh! the ux of workflows is very cool and in the end I was indeed able to build out the Space as I had envisioned!

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions