StrafesNET/maps-service
4
Fork 4
Code Issues 21 Pull Requests Packages Projects Releases Wiki Activity

Change to Proxy Download #208

Merged
Quaternions merged 5 commits from pr1 into staging 2025-07-23 09:17:22 +00:00
Conversation 3 Commits 5 Files Changed 15 +466 -465
Quaternions commented 2025-06-24 12:46:37 +00:00
Owner
Copy Link

Map button is using direct download, but this way the file name can be set which is VERY convenient. It's technically slower and more prone to failure, but maps are no bigger than ~20MB.

  • Figure out why downloaded file is empty
Map button is using direct download, but this way the file name can be set which is VERY convenient. It's technically slower and more prone to failure, but maps are no bigger than ~20MB. - [x] Figure out why downloaded file is empty
Quaternions added 5 commits 2025-06-24 12:46:38 +00:00
ai thinking Rust while doing golang lol a0e5ca4237
openapi: change to proxy download a25e6641f7
openapi: generate 1d86a2af34
submissions: change to proxy download
All checks were successful
continuous-integration/drone/push Build is passing
Details
dc8e3454cf
web: change to proxy download
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
215b36c75a
Quaternions added 1 commit 2025-06-24 12:49:58 +00:00
web: fixups
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
c379177a40
Quaternions added 1 commit 2025-06-24 12:59:26 +00:00
submissions: do the copy
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
168dfbfa74 
the body is closed when DownloadAsset scope ends before it is read
itzaname requested changes 2025-06-27 06:18:18 +00:00
pkg/roblox/asset_location.go Outdated
@@ -71,2 +71,4 @@
return &info, nil
}
func (c *Client) DownloadAsset(info *AssetLocationInfo) ([]byte, error) {
itzaname commented 2025-06-27 06:13:27 +00:00
Owner
Copy Link

Instead of reading the entire stream into memory, return the reader (resp.Body) from this function to use in ogen.

Instead of reading the entire stream into memory, return the reader (resp.Body) from this function to use in ogen.
Quaternions commented 2025-06-27 09:55:30 +00:00
Author
Owner
Copy Link

This is the first thing I tried, but I expected that it would not work because I assumed that resp.Body.Close() had to be called after the body was read. I guess you don't actually need to call it then?

This is the first thing I tried, but I expected that it would not work because I assumed that resp.Body.Close() had to be called after the body was read. I guess you don't actually need to call it then?
itzaname commented 2025-06-27 14:18:41 +00:00
Owner
Copy Link

maps-service/pkg/api/oas_response_encoders_gen.go Line 275 in 6dae7b7493
if closer, ok := response.Data.(io.Closer); ok {

https://git.itzana.me/StrafesNET/maps-service/src/commit/6dae7b7493a386d24881cc36f16d8e7bb366007a/pkg/api/oas_response_encoders_gen.go#L275
Quaternions marked this conversation as resolved
pkg/service/maps.go Outdated
@@ -100,0 +99,4 @@
// download the complete file
asset, err := svc.Roblox.DownloadAsset(info)
ok.Data = bytes.NewReader(asset)
itzaname commented 2025-06-27 06:17:09 +00:00
Owner
Copy Link

You're basically reading all the data in DownloadAsset into a byte array, then turning around and throwing it back into a reader to do the whole process again. Have DownloadAsset return the reader from the request body and you get the following:

  • Less memory copies of the same data
  • Ogen internally uses io.Copy which will stream a Read to a Writer with a buffer instead of reading the whole file out to memory (more efficient)
  • Ogen also closes the Reader on your behalf
You're basically reading all the data in DownloadAsset into a byte array, then turning around and throwing it back into a reader to do the whole process again. Have DownloadAsset return the reader from the request body and you get the following: - Less memory copies of the same data - Ogen internally uses io.Copy which will stream a Read to a Writer with a buffer instead of reading the whole file out to memory (more efficient) - Ogen also closes the Reader on your behalf
Quaternions commented 2025-06-27 09:56:10 +00:00
Author
Owner
Copy Link

Great. I didn't know it would call close for me.

Great. I didn't know it would call close for me.
Quaternions marked this conversation as resolved
web/src/app/maps/[mapId]/page.tsx Outdated
@@ -146,3 +132,1 @@
} finally {
setDownloading(false);
}
await delay(100); // Wait 100ms
itzaname commented 2025-06-27 06:17:55 +00:00
Owner
Copy Link

What is this for

What is this for
Quaternions commented 2025-06-27 09:58:51 +00:00
Author
Owner
Copy Link

Trusting the client not to spam call the endpoint. I repurposed the function that was already there on a whim, it's not important

Trusting the client not to spam call the endpoint. I repurposed the function that was already there on a whim, it's not important
Quaternions marked this conversation as resolved
Quaternions added 1 commit 2025-06-27 10:00:02 +00:00
Revert "submissions: do the copy"
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
afa9b14c75 
This reverts commit 168dfbfa74.
Quaternions added 1 commit 2025-06-27 10:01:27 +00:00
submissions: caller must close body
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
a15e8cef98
Quaternions added 1 commit 2025-06-27 10:06:45 +00:00
web: remove debounce logic
All checks were successful
continuous-integration/drone/push Build is passing
Details
continuous-integration/drone/pr Build is passing
Details
6dae7b7493
Quaternions commented 2025-06-27 10:15:54 +00:00
Author
Owner
Copy Link

The files are empty with 0 bytes, that's what I was trying to fix by copying from the reader before as well

The files are empty with 0 bytes, that's what I was trying to fix by copying from the reader before as well
Quaternions force-pushed pr1 from 6dae7b7493 to a05a20e5b3 2025-06-30 11:05:53 +00:00 Compare
Quaternions force-pushed pr1 from a05a20e5b3 to 592a48e100 2025-07-23 09:02:12 +00:00 Compare
Quaternions force-pushed pr1 from 592a48e100 to 81bb721614 2025-07-23 09:07:56 +00:00 Compare
Quaternions force-pushed pr1 from 81bb721614 to 5d04e1cb5b 2025-07-23 09:09:11 +00:00 Compare
Quaternions force-pushed pr1 from 5d04e1cb5b to 3e353b2ec6 2025-07-23 09:10:44 +00:00 Compare
Quaternions commented 2025-07-23 09:13:55 +00:00
Author
Owner
Copy Link

Apparently this works great.
I think I was using the wrong roblox api key during local testing, and forgot to add

if err != nil{
	return ok, err
}

so I never saw the api error 👍

Apparently this works great. I think I was using the wrong roblox api key during local testing, and forgot to add ```go if err != nil{ return ok, err } ``` so I never saw the api error 👍
Quaternions merged commit 76903656c2 into staging 2025-07-23 09:17:22 +00:00
Quaternions deleted branch pr1 2025-07-23 09:17:22 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
itzaname
Labels
Clear labels
backend
Submissions api related
 bug
Something is not working
 duplicate
This issue or pull request already exists
 enhancement
New feature
 frontend
Website related
 help wanted
Need some help
 invalid
Something is wrong
 question
More information is needed
 validator
Map validation related
 wontfix
This won't be fixed
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
aidan9382 CI dowoge ic3w0lf22 interpreterK itzaname Quaternions
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: StrafesNET/maps-service#208
Delete Branch "pr1"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?