feat(file-storage) File Storage Java SDK#99
Open
kowser-orkes wants to merge 1 commit into
Open
Conversation
6b84835 to
d9e5a2a
Compare
8b668e1 to
e9b37db
Compare
v1r3n
reviewed
Apr 28, 2026
|
|
||
| import com.netflix.conductor.client.http.ConductorClient; | ||
|
|
||
| public class OrkesFileClient extends FileClient { |
Contributor
There was a problem hiding this comment.
rename this to just FileClient please
622beb1 to
121b369
Compare
121b369 to
a05becc
Compare
a05becc to
1c5720f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request type
NOTE: Please remember to run
./gradlew spotlessApplyto fix any format violations.Changes in this PR
Summary
Adds file storage support to the Java SDK, letting workers upload and download large files through Conductor without passing bytes through the server.
FileHandler— developer-facing file reference; wraps a local file before upload (LocalFileHandler) or a server-assignedconductor://file/<fileId>after upload (ManagedFileHandler)FileUploader— developer-facing upload API; implemented byWorkflowFileClient, which binds aworkflowIdat construction so callers never pass it per-callFileClient— internal REST client composingConductorClient+Map<StorageType, FileStorageBackend>; handles single-part and multipart upload flows, upload confirmation, and download URL fetchingFileStorageBackend— per-backend byte-transfer interface (S3, Azure Blob, GCS, Local); storage type is auto-selected from the server's response — no SDK-sidetypeconfig neededAnnotatedWorkerchanges —FileHandlerparameters are auto-resolved fromconductor://file/strings ininputData;FileHandlerreturn values are stored inoutputDataforTaskRunnerto uploadTaskRunnerchanges — after worker execution, scansoutputDataforLocalFileHandlerentries (nullfileHandleId), uploads each, and replaces with the resultingManagedFileHandler; already-uploaded handlers are left untouchedFileClientandFileClientPropertieswired inConductorClientAutoConfiguration;OrkesClients.getFileClient()follows the existing factory patternKey design decisions
WorkflowFamilyResolverManagedFileHandleruses a predictable cache path (<cacheDir>/<fileId>_<fileName>) so two tasks on the same worker node sharing the samefileIdskip re-download on the second accessFileUploadResponse; the SDK validates it against registered backends at upload time and fails fast with a clear error if none matchesTest plan
FileClientTest— upload flow, multipart, backend mismatch fail-fastManagedFileHandlerTest— lazy download, cache hit, retry, concurrent access viaReentrantLockFileHandlerConverterTest— DTO ↔ handler conversionTaskRunnerFileUploadTest— auto-upload scan, skip already-uploaded, null fileClient guardAnnotatedWorkerFileHandlerTest—FileHandlerinput binding, return value passthroughStubFileStorageBackend— in-memory backend for all SDK unit testsAlternatives considered
Describe alternative implementation you have considered