Skip to content

[Feature Request] Search Improvements For Downloading Multiple Items #182

Description

@jwshields

The idea

Suggestion

In the search command (-s or --search)

  • Allow for either, or both:
    • A range of items to be downloaded (ex: "1-8" one through eight, "1,8" one and eight)
    • "Download all"
  • Allow for downloading multiple items in the same invocation without re-issuing the same search, while keeping pagination

It also seems like the program batches searches/results in sets of 100, but if I am on a page, I can only type the numbers for the items on that page.
Example: I am on page 2, items 10-19, I cannot then download item 4 nor can I download item 23

It appears this can be worked around by using --page-size - by increasing the number of items in the immediate result set, but it does not work around being unable to download items not listed on the current page.

Use case

Currently, it is difficult to run a search and download multiple results.

Even with --run-forever - the same search is re-issued after each download is finished.
This is compounded if you paginate, because the pagination is not kept

  • Run a search with --run-forever
  • Go to the next results page (results 10-19 with the default pagination size)
  • Download an item
  • Download finished
  • run-forever starts the loop again, search is re-issued, result page is for items 1-9

Regarding the batching of results and page sizes, if I set a page size of >100, it will issue multiple search requests to return the results; pair that with --run-forever and it will start being horribly inefficient - it will run the needed searches after each download finishes.

These two things make it much more difficult to search across wide queries.

Additional Info

Download multiple items, don't issue the same search multiple times:

Current Behavior Proposed Behavior
- Issue search
- Result page 1
- Download item 1
- Search is reissued
- Result page 1
- Download item 2
- Issue search
- Result page 1
- Download items 1-8
- Return to results page

Pagination is not kept, run forever re-issues searches when download finishes:

Current Behavior Proposed Behavior
- Start with --run-forever
- Result page 1
- Download item 1
- Search is reissued
- Result page 1
- Download item 2
- Start with --run-forever
- Result page 1
- Download item 1
- Return to results page
- Download item 2
- Start with --run-forever
- Result page 1
- "next"
- Result page 2 (items 10-19 at default pagination)
- Download item 10
- Return to results page
- Search is re-issued
- Result page 1 (items 1-9 at default pagination)
- ❌ Cannot download item 11
- Start with --run-forever
- Result page 1
- "next"
- Result page 2 (items 10-19 at default pagination)
- Download item 10
- Return to results page
- Results page 2 (items 10-19 at default pagination)
- Download item 11

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions