Commit graph

222 commits

Author SHA1 Message Date
Daz DeBoer
6cc033f2b3
Make build-result-capture script compatible with config-cache 2022-06-03 07:12:23 -06:00
Daz DeBoer
7b79b2a752
Report on read-only / write-only cache 2022-06-02 23:16:52 -06:00
Daz DeBoer
143774290e
Add more details to cache summary report 2022-06-02 22:39:55 -06:00
Daz DeBoer
a9a5bcf180
Move writing of cache report into job-summary.ts 2022-06-02 22:39:09 -06:00
Daz DeBoer
f9c8fcf79f
Adding caching details to Job Summary 2022-06-02 14:21:10 -06:00
Daz DeBoer
86e82987ba
Write job summary in post action
- Save build results in file encoded as JSON
- Read all build results in post action and render as table in job summary
2022-06-02 12:58:49 -06:00
Daz DeBoer
8f3c97e3f1
Extract action state-tracking out of caches.ts 2022-06-02 12:20:01 -06:00
Daz DeBoer
d95713bd5d
Extract init scripts into resource files 2022-06-02 11:53:33 -06:00
Daz DeBoer
28b774ebdb
Allow override of sections of cache key
These internal env vars are designed primarily for testing,
but may also prove useful for folks to experiment with more optimal
caching setups.
2022-05-29 09:32:24 -06:00
Daz DeBoer
be62f7d934
Merge branch 'releases/v2.1'
* releases/v2.1:
  Build outputs
  Remove downloaded wrapper zips before caching
  Fix typo in worklow step
2022-05-29 09:23:36 -06:00
Daz DeBoer
f2bb19b43a
Remove downloaded wrapper zips before caching
The wrapper zip files are redundant, and not required after extraction.
Gradle 7.5+ will delete these automatically, but we delete them for older
versions to avoid caching the wrapper distributions twice.

Sinc the `gradle-home-cache-excludes` parameter does not support
wildcards, we remove them explicitly.
2022-05-29 08:49:36 -06:00
Daz DeBoer
7645e6e536
Replace emoji characters with markdown 2022-05-24 16:37:15 +02:00
Daz DeBoer
646074d659
Write build scan link to GHA job summary
The new Job Summary functionality permits better rendering of build
results, including clickable build scan links
2022-05-24 15:01:44 +02:00
Daz DeBoer
d63db6a05e
Remove unclickable buildScan link from failure
Build scan links will be rendered in GHA Job Summary.
2022-05-24 15:01:08 +02:00
Daz DeBoer
0e24fa1975
Save/restore exploded Gradle dist rather than zip
To save space, future versions of Gradle are likely to delete the downloaded distribution
after extracting it. See gradle/gradle#3605 and gradle/gradle#19495.
To cater for this we will now save/restore the extracted distribution rather than the
downloaded zip file.
2022-04-04 21:43:31 -06:00
Daz DeBoer
0d13054264
Log stack trace for cache errors 2022-03-18 13:56:40 -06:00
Daz DeBoer
53e57a851c
Make Gradle Home detection compatible with MacOS
MacOS runners are initialized with a Gradle User Home directory including
the `~/.gradle/notifications` directory. This was causing the action to skip
restoring the Gradle User Home on MacOS.

This fix limits the pre-existing GUH check to the `~/.gradle/caches` directory
which isn't pre-initialized in the runner.

Fixes #155
2022-02-03 09:46:37 -07:00
Daz DeBoer
702bb97ecb
Do not overwrite existing gradle.properties 2022-01-24 15:18:11 -07:00
Daz DeBoer
f75a3e8436
Make build-scan notice a WARNING for failed builds 2022-01-21 08:09:32 -07:00
Daz DeBoer
c349fa5b1f
Do not hide ReserveCacheError failures
When the target entry already exists, we are unable to save. Ensuring this situation
is logged helps make the logs more understandable.
2022-01-21 08:08:32 -07:00
Daz DeBoer
08d5b40ca5
Add experimental support for 'cache-write-only'
There may be cases where it a "fresh" cache entry would be beneficial,
for example if the Gradle User Home cache entry grows over time.

This setting would run the build as if no prior cache entry exists.
2022-01-20 15:04:11 -07:00
Daz DeBoer
0a5ede19a9
Improve cache-reporting when entry already exists 2022-01-20 15:04:11 -07:00
Daz DeBoer
a23ac1d61c
Add experimental support for 'strict-match' caching
By default, the action will attempt to restore a Gradle User Home
cache entry from a different set of matrix inputs (or a different Job entirely)
if an existing entry is not found for the current Job (including matrix inputs).

By specifying the experimental `gradle-home-cache-strict-match` parameter, a user
can avoid this fuzzy matching and ensure that a job execution starts with an
empty Gradle User Home if no entry from a prior execution is found.
2022-01-20 15:03:54 -07:00
Daz DeBoer
7dee0f45c2
Fix check for pre-existing Gradle User Home 2022-01-17 14:44:32 -07:00
Daz DeBoer
50ca2bca83
Improve messages for cache-disabled scenario 2022-01-17 14:44:32 -07:00
Daz DeBoer
3aa7bfe163
Initialize pre-existing Gradle User Home 2022-01-17 14:44:32 -07:00
Daz DeBoer
06d64212d3
Initialize the Gradle User Home even when caching is disabled 2022-01-17 14:29:11 -07:00
Daz DeBoer
42faf281e6
Include names of requested tasks in build scan notice
This should make it easier to differentiate between build scan notices for different
Gradle invocations in the same Job.

Fixes: #124
2021-12-31 10:34:53 -07:00
Daz DeBoer
13d93c1ca1
Check executable bit on Gradle script
Fixes #76
2021-12-30 21:40:27 -07:00
Daz DeBoer
76ea8a76b2
Treat configuration-cache as an extracted entry
Instead of using a fallback strategy to locate a configuration-cache entry
based on the current job and git SHA, these entries are now keyed based on their
file content with the keys persisted in the primary Gradle User Home entry.

This removes the chance of having a configuration-cache entry restored that is
incompatible with the restored Gradle User Home state, and makes the logic easier
to understand.

This change involved a fairly major refactor, with the CacheEntryExtractor being
split out from the primary cache implementation, and adding a separate extractor
implementation for configuration-cache.
2021-12-30 16:03:02 -07:00
Daz DeBoer
12fc52a49a
Combine java-toolchain entry definitions
- Remove configuration parameter for cache-entry-definitions. This allows richer modelling of extracted entries.
- Include both *.zip and *.tar.gz in entry definition for java-toolchains.
2021-12-30 16:03:02 -07:00
Daz DeBoer
c157d0a332
Cache downloaded Java Toolchains
Due to limitations with the way extracted entries are defined, a separate entry
is defined for *nix and windows.

Fixes #33
2021-12-30 16:02:02 -07:00
Daz DeBoer
0e8b9655a0
Allow action to be used without Gradle invocation
If the user supplies no 'arguments' parameter, the action will function as a
'setup-gradle' action, adding Gradle to the PATH and enabling other features
without actually running a Gradle build.

Any subsequent Gradle invocations in the workflow will benefit from:
- Save/restore of Gradle User Home
- Save/restore of configuration-cache data
- Capture of build-scan URLs

These features are enabled via Gradle User Home, so any Gradle invocation that
uses the same Gradle User Home will be included.
2021-12-17 10:29:58 -07:00
Daz DeBoer
e1f84aa44d
Restore/save configuration-cache data in first action
Previously, the action was restoring/saving the configuration-cache data for each
step that applied the action. In order to support Gradle invocations that are _not_
managed by the action, the configuration-cache restore is now performed in the initial
action step, and save is performed in the final post-action step.

The build root directories are recorded for each invocation via an init script.
2021-12-08 15:09:27 -07:00
Daz DeBoer
1041604f29
Ensure save/restore only on first action step
Instead of relying on the separate cache implementations to check for the
existence of cached products, we now explicitly track whether or not the execution
is the first time the action has been invoked for a job.
2021-12-08 15:09:27 -07:00
Daz DeBoer
253d6427fd
Extract cache-reporting into separate file
Cache reporting is self-contained enough that it deserves some separation.
2021-12-08 15:09:26 -07:00
Daz DeBoer
887e1a09ab
Include original error in log messages 2021-12-08 14:44:57 -07:00
Daz DeBoer
37f2880a8a Cache wrapper zips and generated jars individually
Using a single cache entry for all files of a type is necessary to avoid
overloading the cache service. However, this mechanism is not very efficient
for certain artifacts like wrapper zips and generated-gradle-jars, where the
same individual files are often shared between different jobs.

With this change, any configured file patterns that do not end in '*' will
be cached as individual files. At this time this includes downloaded wrapper
zips and generated-gradle-jars.

Fixes #78
2021-12-08 14:05:13 -07:00
Daz DeBoer
b88c4086b9
Don't overwrite title in build-scan notice 2021-12-08 13:07:52 -07:00
Daz DeBoer
322805e800
Refactor: use a single .json file to describe all cached artifact bundles (#121)
This is a pure refactor, moving from a separate .cache file per bundle to a single cache-metadata.json file describing all bundles. Instead of storing cache metadata in a separate .cache file per artifact bundle, all of the metadata is now stored in a single `.json` file.

This will make it easier to implement more flexible artifact-caching strategies, such as caching each wrapper zip separately.

* Always include cache protocol version in cache key
* Store all cache metadata in a single JSON file
* Rename cache-metadata file and bump protocol version
* Polish and documentation
2021-11-28 10:19:56 -07:00
Daz DeBoer
a25c0ce6cb
Improve title on build scan notice 2021-11-28 08:04:54 -07:00
Daz DeBoer
76f85a724c
Record build scan url directly from init script
Instead of writing the URL to a file on disk, reading it later and
using the Actions API to record the output parameter and write the notice,
these things are now done directly via Actions commands emitted directly
from the init script.
2021-11-27 20:49:10 -07:00
Daz DeBoer
45ef022607
Always initialize Gradle User Home with init script
The generated gradle.properties file and init script are not cached,
so need to be generated even when restoring from cache.
2021-11-27 20:45:49 -07:00
Daz DeBoer
a72af0b6a6
Persist build-scan-capture init script into Gradle User Home
Instead of writing this file to a temp directory and referencing it
on the Gradle command line, the init script is now written to Gradle
User Home so that it is picked up automatically.
2021-11-27 20:44:43 -07:00
Daz DeBoer
b0c29bffb7
Use a properties file to disable daemon execution
Instead of passing `--no-daemon` on the command line, the same
functionality is now acheived by writing a gradle.properties file
when initializing Gradle User Home.
2021-11-27 16:07:07 -07:00
Daz DeBoer
996094e8e8
Avoid failing job on any failure in post-action
Failures to store cache entries should not fail the action or the Job.
This fix attempts to catch and log any unexpected errors that occur when
saving cache entries.

Fixes: #119
Fixes: #120
2021-11-15 09:31:56 -07:00
Daz DeBoer
4137be6a8b
Minor improvement to logging in post-action
- Ensure that "Caching Gradle state" group always has 1 message
- Only print cache report when entries were restored or saved
2021-11-05 08:35:45 -06:00
Daz DeBoer
4e899835b3
Avoid failing build on distributions cache errors
- Warn and continue on failure to restore a Gradle distribution from cache
- Warn and continue on failure to save a Gradle distribution to cache
- Extract common functionality for consistent handling of cache failures

Fixes #116
2021-11-05 07:01:48 -06:00
Daz DeBoer
717db318c1
Fix logging of cache key for Gradle User Home 2021-10-31 20:49:29 -06:00
Daz DeBoer
472ac8a356
Report sizes of cache entries
Using the patched version of @actions/cache, we now report the total
size of cache entries restored/saved, as well as details of each one.
2021-10-30 13:45:39 -06:00
Daz DeBoer
d785346c8c
Use cache.description 2021-10-30 13:44:22 -06:00
Daz DeBoer
6ca4d4ade2
Improve formatting for caching report 2021-10-30 12:15:00 -06:00
Daz DeBoer
75cec40e58
Rename 'report' classes to 'listener'
- `CachingReport` -> `CacheListener`
- `CacheEntryReport` -> `CacheEntryListener`
2021-10-30 12:15:00 -06:00
Daz DeBoer
c317ccac62
Refactor: extract cache-base out of cache-utils 2021-10-30 12:15:00 -06:00
Daz DeBoer
a74bb0fad6
Monitor cache saves and add basic caching report
- Restore `CachingReport` instance in 'post' action
- Record keys for any entries saved
- Report caching activity as JSON in post action
2021-10-30 12:14:59 -06:00
Daz DeBoer
727b4612ba
Fix early return from bundle processing 2021-10-30 12:11:59 -06:00
Daz DeBoer
f0f68e07c3
Better handling of .cache files that are not restored
- Remove any .cache file that is not restored
- Report on any .cache file that exists but has no config
2021-10-30 06:28:20 -06:00
Daz DeBoer
8ba5a0033b
Track 'fully-restored' by tracking each cache restore
Instead of tracking a single 'fully-restored' flag, track the restore status of each
cache entry restore. If any of these are requested but not restored, then the overall
Gradle User Home cache is not fully restored.

Added special handling for the case when zero artifact bundles are set: this is used
in tests to simulate a not-fully-restored state.
2021-10-30 06:05:21 -06:00
Daz DeBoer
9edc2a11bd
Only restore configuration-cache if Gradle Home is fully restored
Fixes #107
2021-10-29 09:34:36 -06:00
Daz DeBoer
4ebd000afd
Bundle all downloaded dependency files
Previously, only .jar files were bundled, with other files (modules, POMs, zips, etc)
being left in Gradle User Home. All downloaded files are now included in the bundle.

Fixes #100
2021-10-29 08:03:03 -06:00
Daz DeBoer
063fc6a872
Allow source files to contain lines up to 120 characters
This avoids excessive line-feeds when reformatting code to 80 char lines.
2021-10-29 07:50:06 -06:00
Daz DeBoer
e3ada7e5c2
Use multiline input parameters instead of JSON input
The `gradle-home-cache-includes` and `gradle-home-cache-excludes` parameters were initially implemented
as JSON string inputs. This makes these inputs non-idiomatic and easier to get wrong.

This change converts them to multi-line input parameters.

Fixes #106
2021-10-29 07:29:57 -06:00
Daz DeBoer
27f2dc276c
Rename and document parameters for clarity
- cache-paths -> gradle-home-cache-includes
- cache-exclude-paths -> gradle-home-cache-excludes
- CACHE_DEBUG_ENABLED -> GRADLE_BUILD_ACTION_CACHE_DEBUG_ENABLED
2021-10-27 16:05:07 -06:00
Daz DeBoer
39db90e99b
Include build scan link in failure notice 2021-10-21 12:04:25 -06:00
Daz DeBoer
4cf255df10
Move bundle metadata files out of caches directory
Instead, use a separate '.gradle-build-action' directory for bundle
metadata files. This directory is always part of the cache-path.
2021-10-21 11:13:09 -06:00
Daz DeBoer
1113cb87cb
Allow cache-exclude-paths to be configured for action
All excluded paths are deleted prior to caching the Gradle User Home.
2021-10-21 11:13:09 -06:00
Daz DeBoer
9c95294209
Allow cache-paths to be set via action config 2021-10-21 11:13:08 -06:00
Daz DeBoer
f901ec9c20
Bump cache-protocol version 2021-10-20 15:04:14 -06:00
Daz DeBoer
a94b9252d5
Improve cache logging 2021-10-16 10:15:40 -06:00
Daz DeBoer
cb6a0acca4
Use precise matching for artifact bundles
This should fix the warnings issued when saving artifact bundles.
2021-10-16 09:49:15 -06:00
Daz DeBoer
aa2ed2e033
Use cache protocol version for bundle keys too 2021-10-16 09:49:14 -06:00
Daz DeBoer
263f84178a
Prefix cache key with protocol version
This will ensure that incompatiblee cache entries generated by previous action releases
will not be used.
2021-10-16 08:33:42 -06:00
Daz DeBoer
709ded51a5
Treat directory for instrumented jar as single artifact
Leaving the `.lock` and `.receipt` files lying around was causing
issues when the actual jar files were not restored. Now the entire
directory will either be missing, or completely restored.
2021-10-15 14:54:29 -06:00
Daz DeBoer
da64595ccc
Make artifact bundle definitions an input parameter 2021-10-15 13:21:13 -06:00
Daz DeBoer
29b14c7fca
Refactor: rename methods for 'bundle' concept 2021-10-15 12:34:38 -06:00
Daz DeBoer
53ccc3e0d7
Add more cache debug logging 2021-10-15 11:45:15 -06:00
Daz DeBoer
8ab7c9d8dd
Do not fail action on cache errors
Ensure that we catch and log errors in `beforeSave` and `afterRestore`,
and do not fail the entire workflow in these cases.
2021-10-14 12:19:24 -06:00
Daz DeBoer
4968d2280b
Allow time for processes to release file locks on windows 2021-10-04 23:59:08 +02:00
Daz DeBoer
60b1ffac6b
Fix build-scan-init script to work with build-in build-scan plugins 2021-09-29 14:39:48 -06:00
Daz DeBoer
0918f5f2a4
Rename source file for clarity 2021-09-28 00:04:50 -06:00
Daz DeBoer
a7174b82a2
Use init script to capture build scan URL
Instead of parsing the log output, we instead register a
buildScanPublished listener and record the build scan URL
to a file. This file is subsequently read to report the
build scan URL.

Fixes #30
2021-09-28 00:04:41 -06:00
Daz DeBoer
5a5a5b4387
Normalize paths to Gradle User Home when calculating cache keys
Fixes #77
2021-09-27 21:52:03 -06:00
Daz DeBoer
cc5cdb7fe0
Adapt caching for GRADLE_USER_HOME env var
Fixes #74
2021-09-27 21:50:14 -06:00
Daz DeBoer
bebb162342
Usability improvements:
- Include bundle name in cache key
- Emit a few more messages at 'info'
2021-09-15 17:50:53 -06:00
Daz DeBoer
6084a4eb65
Cache artifacts with single entry per type
When caching is too fine-grained, an excessive number of cache
requests can result in HTTP 429 errors due to rate limiting.
By caching all artifacts of a particular type in a single entry
we hope to mitigate this, at the expense of some reduction in
cache space optimization.

This change also adds caching for all dependency jars, as well as
instrumented jars in the 'caches/jars-X' directory.
2021-09-15 17:45:47 -06:00
Daz DeBoer
dbb485d80d
Only perform post-restore and pre-save cache actions when required 2021-09-15 17:45:47 -06:00
Daz DeBoer
fe64d05f86
Do not cache dependency jars individually
Unfortunately, doing this overloads the GitHub actions cache infrastructure
leading to failures and unpredictable results.
A later solution may re-implement artifact sharing for dependency jars
as well as jars within the `caches/jars-9` directory. But for now these
will be duplicated across each Gradle User Home cache entry.
2021-09-14 18:07:28 -06:00
Daz DeBoer
decca791c5
Consolidate error handling for cache restore/save 2021-09-14 13:39:37 -06:00
Daz DeBoer
bd08e7b7cd
Do not fail build on error restoring artifact for marker
Instead, catch and report these errors before continuing.
2021-09-14 13:38:48 -06:00
Daz DeBoer
cca55d0890
Fail action execution on unhandled errors
Without this, the error logs contain an "UnhandledPromiseRejectionError"
but the action is reported as succeeding.
2021-09-14 07:48:06 -06:00
Daz DeBoer
bbe1574290
Add some TODOs to the code for future reference 2021-09-13 11:50:07 -06:00
Daz DeBoer
4264cda558
Group log messages generated when provisioning Gradle 2021-09-13 11:50:07 -06:00
Daz DeBoer
3390540145
Simplify setting caches to disabled or read-only 2021-09-13 11:04:42 -06:00
Daz DeBoer
1c72a31463
Allow cache debug logging to be selectively enabled
- Move more messages out of main 'info' log
- Only log cache entry size when debugging
- Process cache entries sequentially when debugging
2021-09-13 10:52:09 -06:00
Daz DeBoer
7dfbe33bba
Improve formatting for cache entry size reporting 2021-09-13 10:51:24 -06:00
Daz DeBoer
e63ddf9c00
Avoid warning for cache reservation errors
We have no way of knowing if another build has already cached these
artifacts, so CacheReservationError is expected.
2021-09-13 10:50:47 -06:00
Daz DeBoer
cae99bf6d9
Tidy up cache entry de-duplication
- Rename feature to "common artifact caching"
- Cleanup logging
- Refactor/rename for clarity
2021-09-12 10:31:52 -06:00
Daz DeBoer
5a90152b1f
Restore/cache deduplicated files in parallel 2021-09-12 10:31:52 -06:00
Daz DeBoer
4b92b8d013
Cache downloaded dependency jars separately
This will further remove common files out the the Gradle User Home
cache entries, reducing cache usage by removing redundancy.
2021-09-12 10:31:51 -06:00
Daz DeBoer
693293c29a
Consolidate logic for separate artifact file caching
Making this functionality more general will make it easier to add other
artifacts to the set that we cache separately.
2021-09-12 10:31:51 -06:00