-
-
Notifications
You must be signed in to change notification settings - Fork 34.7k
src: consolidate C++ ReadFileSync/WriteFileSync utilities #61662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This patch moves `ReadFileSync` and `WriteFileSync` from `src/util.cc` to `src/node_file_utils.cc`, consolidates the implementation to reuse code, and adds a few more enhancements: For `ReadFileSync`: - Use fstat-based pre-allocation to minimize buffer resizing and repeated reads for bigger files. - Handle various potential overflows in size conversions. - Handle fallback for 0-byte special files. For `WriteFileSync`: - Handle potential partial writes for big enough files and support non-seekable files (with -1 as offset). - Handle 0-byte writes correctly. In both cases, this now avoids hard aborts and return error code for the caller to handle as much as possible, except `std::vector<char> ReadFileSync(FILE* fp)` which is part of the embedder API. This patch uses the new `ReadFileSync` to address a TODO in node_sea.bin.cc.
|
Review requested:
|
|
cc @nodejs/cpp-reviewers |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #61662 +/- ##
==========================================
- Coverage 89.74% 89.72% -0.03%
==========================================
Files 674 675 +1
Lines 204348 204472 +124
Branches 39271 39290 +19
==========================================
+ Hits 183396 183460 +64
- Misses 13262 13303 +41
- Partials 7690 7709 +19
🚀 New features to boost your workflow:
|
Qard
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. A few small code quality improvement ideas, but I'm fine landing as-is. These are just minor suggestions to simplify things slightly. 🙂
Co-authored-by: Stephen Belanger <[email protected]>
|
Thanks for the suggestion. @Qard @RafaelGSS Can you take a look again? |
This patch moves `ReadFileSync` and `WriteFileSync` from `src/util.cc` to `src/node_file_utils.cc`, consolidates the implementation to reuse code, and adds a few more enhancements: For `ReadFileSync`: - Use fstat-based pre-allocation to minimize buffer resizing and repeated reads for bigger files. - Handle various potential overflows in size conversions. - Handle fallback for 0-byte special files. For `WriteFileSync`: - Handle potential partial writes for big enough files and support non-seekable files (with -1 as offset). - Handle 0-byte writes correctly. In both cases, this now avoids hard aborts and return error code for the caller to handle as much as possible, except `std::vector<char> ReadFileSync(FILE* fp)` which is part of the embedder API. This patch uses the new `ReadFileSync` to address a TODO in node_sea.bin.cc. PR-URL: #61662 Reviewed-By: Rafael Gonzaga <[email protected]> Reviewed-By: Stephen Belanger <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>
|
Landed in 9bc369f...3ab9dd8 |
This patch moves `ReadFileSync` and `WriteFileSync` from `src/util.cc` to `src/node_file_utils.cc`, consolidates the implementation to reuse code, and adds a few more enhancements: For `ReadFileSync`: - Use fstat-based pre-allocation to minimize buffer resizing and repeated reads for bigger files. - Handle various potential overflows in size conversions. - Handle fallback for 0-byte special files. For `WriteFileSync`: - Handle potential partial writes for big enough files and support non-seekable files (with -1 as offset). - Handle 0-byte writes correctly. In both cases, this now avoids hard aborts and return error code for the caller to handle as much as possible, except `std::vector<char> ReadFileSync(FILE* fp)` which is part of the embedder API. This patch uses the new `ReadFileSync` to address a TODO in node_sea.bin.cc. PR-URL: #61662 Reviewed-By: Rafael Gonzaga <[email protected]> Reviewed-By: Stephen Belanger <[email protected]> Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: James M Snell <[email protected]>
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [node](https://nodejs.org) ([source](https://github.com/nodejs/node)) | patch | `25.6.0` → `25.6.1` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>nodejs/node (node)</summary> ### [`v25.6.1`](https://github.com/nodejs/node/releases/tag/v25.6.1): 2026-02-10, Version 25.6.1 (Current), @​aduh95 [Compare Source](nodejs/node@v25.6.0...v25.6.1) ##### Notable Changes - \[[`47df4328d7`](nodejs/node@47df4328d7)] - **build,deps**: replace `cjs-module-lexer` with `merve` (Yagiz Nizipli) [#​61456](nodejs/node#61456) ##### Commits - \[[`47df4328d7`](nodejs/node@47df4328d7)] - **build,deps**: replace cjs-module-lexer with merve (Yagiz Nizipli) [#​61456](nodejs/node#61456) - \[[`a727054503`](nodejs/node@a727054503)] - **deps**: upgrade npm to 11.9.0 (npm team) [#​61685](nodejs/node#61685) - \[[`c78c49ed6b`](nodejs/node@c78c49ed6b)] - **deps**: update amaro to 1.1.7 (Node.js GitHub Bot) [#​61730](nodejs/node#61730) - \[[`4790816d9b`](nodejs/node@4790816d9b)] - **deps**: update minimatch to 10.1.2 (Node.js GitHub Bot) [#​61732](nodejs/node#61732) - \[[`8c71740e8a`](nodejs/node@8c71740e8a)] - **deps**: update undici to 7.21.0 (Node.js GitHub Bot) [#​61683](nodejs/node#61683) - \[[`e559ef6ab1`](nodejs/node@e559ef6ab1)] - **deps**: update googletest to [`56efe39`](nodejs/node@56efe39) (Node.js GitHub Bot) [#​61605](nodejs/node#61605) - \[[`300de2bb5a`](nodejs/node@300de2bb5a)] - **deps**: update amaro to 1.1.6 (Node.js GitHub Bot) [#​61603](nodejs/node#61603) - \[[`e71e9505ef`](nodejs/node@e71e9505ef)] - **dns**: fix Windows SRV ECONNREFUSED by adjusting c-ares fallback detection (notvivek12) [#​61453](nodejs/node#61453) - \[[`439b816bc7`](nodejs/node@439b816bc7)] - **doc**: clarify EventEmitter error handling in threat model (Matteo Collina) [#​61701](nodejs/node#61701) - \[[`c1c6641f23`](nodejs/node@c1c6641f23)] - **doc**: mention default option for test runner env (Steven) [#​61659](nodejs/node#61659) - \[[`41ec451f98`](nodejs/node@41ec451f98)] - **doc**: fix --inspect security warning section (Tim Perry) [#​61675](nodejs/node#61675) - \[[`bb90ef2356`](nodejs/node@bb90ef2356)] - **doc**: document `url.format(urlString)` as deprecated under DEP0169 (René) [#​61644](nodejs/node#61644) - \[[`513df82e6f`](nodejs/node@513df82e6f)] - **doc**: update to Visual Studio 2026 manual install (Mike McCready) [#​61655](nodejs/node#61655) - \[[`9409d30736`](nodejs/node@9409d30736)] - **doc**: deprecation add more codemod (Augustin Mauroy) [#​61642](nodejs/node#61642) - \[[`75a7a67151`](nodejs/node@75a7a67151)] - **doc**: fix grammatical error in README.md (ayj8201) [#​61653](nodejs/node#61653) - \[[`821e59e884`](nodejs/node@821e59e884)] - **doc**: correct tools README Boxstarter link (Mike McCready) [#​61638](nodejs/node#61638) - \[[`4998f539a0`](nodejs/node@4998f539a0)] - **doc**: update `server.dropMaxConnection` link (YuSheng Chen) [#​61584](nodejs/node#61584) - \[[`9383ac4ab7`](nodejs/node@9383ac4ab7)] - **http**: implement slab allocation for HTTP header parsing (Mert Can Altin) [#​61375](nodejs/node#61375) - \[[`e90eb1d561`](nodejs/node@e90eb1d561)] - **meta**: persist sccache daemon until end of build workflows (René) [#​61639](nodejs/node#61639) - \[[`ade36ac367`](nodejs/node@ade36ac367)] - **meta**: bump github/codeql-action from 4.31.9 to 4.32.0 (dependabot\[bot]) [#​61622](nodejs/node#61622) - \[[`26638bd67f`](nodejs/node@26638bd67f)] - **meta**: bump step-security/harden-runner from 2.14.0 to 2.14.1 (dependabot\[bot]) [#​61621](nodejs/node#61621) - \[[`eaa9a96cb6`](nodejs/node@eaa9a96cb6)] - **meta**: bump actions/setup-python from 6.1.0 to 6.2.0 (dependabot\[bot]) [#​61627](nodejs/node#61627) - \[[`fd98187828`](nodejs/node@fd98187828)] - **meta**: bump cachix/cachix-action (dependabot\[bot]) [#​61626](nodejs/node#61626) - \[[`820c1d021c`](nodejs/node@820c1d021c)] - **meta**: bump actions/setup-node from 6.1.0 to 6.2.0 (dependabot\[bot]) [#​61625](nodejs/node#61625) - \[[`72a4136bd5`](nodejs/node@72a4136bd5)] - **meta**: bump actions/cache from 5.0.1 to 5.0.3 (dependabot\[bot]) [#​61624](nodejs/node#61624) - \[[`e3ef6cb3bc`](nodejs/node@e3ef6cb3bc)] - **meta**: bump peter-evans/create-pull-request from 8.0.0 to 8.1.0 (dependabot\[bot]) [#​61623](nodejs/node#61623) - \[[`020a836202`](nodejs/node@020a836202)] - **meta**: bump actions/stale from 10.1.0 to 10.1.1 (dependabot\[bot]) [#​61620](nodejs/node#61620) - \[[`0df72f07c8`](nodejs/node@0df72f07c8)] - **meta**: bump actions/checkout from 6.0.1 to 6.0.2 (dependabot\[bot]) [#​61619](nodejs/node#61619) - \[[`d147c08b83`](nodejs/node@d147c08b83)] - **module**: do not invoke resolve hooks twice for imported cjs (Joyee Cheung) [#​61529](nodejs/node#61529) - \[[`a2843f8556`](nodejs/node@a2843f8556)] - **net**: defer synchronous destroy calls in internalConnect (RajeshKumar11) [#​61658](nodejs/node#61658) - \[[`7fb7030781`](nodejs/node@7fb7030781)] - **repl**: fix flaky test-repl-programmatic-history (Matteo Collina) [#​61614](nodejs/node#61614) - \[[`d4c9b5cf5b`](nodejs/node@d4c9b5cf5b)] - **sqlite**: avoid extra copy for large text binds (Ali Hassan) [#​61580](nodejs/node#61580) - \[[`aa1b3661d9`](nodejs/node@aa1b3661d9)] - **sqlite**: use DictionaryTemplate for run() result (Mert Can Altin) [#​61432](nodejs/node#61432) - \[[`9c8ad7e881`](nodejs/node@9c8ad7e881)] - **src**: elide heap allocation in structured clone implementation (Anna Henningsen) [#​61703](nodejs/node#61703) - \[[`c4ecfef93d`](nodejs/node@c4ecfef93d)] - **src**: use simdutf for one-byte string UTF-8 write in stringBytes (Mert Can Altin) [#​61696](nodejs/node#61696) - \[[`28905b9734`](nodejs/node@28905b9734)] - **src**: consolidate C++ ReadFileSync/WriteFileSync utilities (Joyee Cheung) [#​61662](nodejs/node#61662) - \[[`e90cec2f69`](nodejs/node@e90cec2f69)] - **test**: restraint version replacement pattern in snapshots (Chengzhong Wu) [#​61748](nodejs/node#61748) - \[[`adce20c0a1`](nodejs/node@adce20c0a1)] - **test**: print stack immediately avoiding GC interleaving (Chengzhong Wu) [#​61699](nodejs/node#61699) - \[[`7643bc8999`](nodejs/node@7643bc8999)] - **test**: fix case-insensitive path matching on Windows (Matteo Collina) [#​61682](nodejs/node#61682) - \[[`23d1ecf66f`](nodejs/node@23d1ecf66f)] - **test**: fix flaky test-performance-eventloopdelay (Matteo Collina) [#​61629](nodejs/node#61629) - \[[`99012a88ed`](nodejs/node@99012a88ed)] - **test**: remove duplicate wpt tests (Filip Skokan) [#​61617](nodejs/node#61617) - \[[`a8b32b8ce1`](nodejs/node@a8b32b8ce1)] - **test**: fix race condition in watch mode tests (Matteo Collina) [#​61615](nodejs/node#61615) - \[[`086a5a5a25`](nodejs/node@086a5a5a25)] - **test**: update WPT for url to [`e3c46fd`](nodejs/node@e3c46fdf55) (Node.js GitHub Bot) [#​61602](nodejs/node#61602) - \[[`f0574fd419`](nodejs/node@f0574fd419)] - **test**: use the skipIfNoWatch() utility function (Luigi Pinca) [#​61531](nodejs/node#61531) - \[[`b064ddc221`](nodejs/node@b064ddc221)] - **test**: unify assertSnapshot common patterns (Chengzhong Wu) [#​61590](nodejs/node#61590) - \[[`17122e521b`](nodejs/node@17122e521b)] - **test\_runner**: fix test enqueue when test file has syntax error (Edy Silva) [#​61573](nodejs/node#61573) - \[[`bad3f02dd9`](nodejs/node@bad3f02dd9)] - **tools**: enforce removal of `lts-watch-*` labels on release proposals (Antoine du Hamel) [#​61672](nodejs/node#61672) - \[[`a8f33fd6bd`](nodejs/node@a8f33fd6bd)] - **tools**: use ubuntu-slim runner in meta GitHub Actions (Tierney Cyren) [#​61663](nodejs/node#61663) - \[[`c843e447ca`](nodejs/node@c843e447ca)] - **tools**: test `--shared-merve` in `test-shared` workflow (Antoine du Hamel) [#​61649](nodejs/node#61649) - \[[`2fedc03f96`](nodejs/node@2fedc03f96)] - **tools**: update OpenSSL to 3.5.5 in `test-shared` (Antoine du Hamel) [#​61551](nodejs/node#61551) - \[[`1c1db94670`](nodejs/node@1c1db94670)] - **tools,win**: upgrade install additional tools to Visual Studio 2026 (Mike McCready) [#​61562](nodejs/node#61562) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi45Ny4wIiwidXBkYXRlZEluVmVyIjoiNDIuOTcuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90IiwiYXV0b21hdGlvbjpib3QtYXV0aG9yZWQiLCJkZXBlbmRlbmN5LXR5cGU6OnBhdGNoIl19-->
This patch moves
ReadFileSyncandWriteFileSyncfromsrc/util.cctosrc/node_file_utils.cc, consolidates the implementation to reuse code, and adds a few more enhancements:For
ReadFileSync:For
WriteFileSync:In both cases, this now avoids hard aborts and return error code for the caller to handle as much as possible, except
std::vector<char> ReadFileSync(FILE* fp)which is part of the embedder API.This patch uses the new
ReadFileSyncto address a TODO in node_sea.bin.cc.Some numbers from
--build-sea(note that most of the time is spent on parsing the executable and building the SEA, but even so there's a measurable difference just from the file I/O because the binary is ~120MB).