Skip to content

feat(source): delete moves to trash#170

Open
bosschaert wants to merge 16 commits intomainfrom
trash
Open

feat(source): delete moves to trash#170
bosschaert wants to merge 16 commits intomainfrom
trash

Conversation

@bosschaert
Copy link
Copy Markdown
Contributor

@bosschaert bosschaert commented Apr 2, 2026

  • Deleting a document moves it to the .trash folder at the root of the site storage. If there is already a document with the same name a unique suffix is added to the document name.
  • Deleting a folder moves all of its contents recursively to the .trash under a folder with the same name. If a folder with this name already exists in the trash a unique suffix is added.

Each document moved to the trash has a new metadata property 'doc-path' with the path it was originally located at.

Also included here, the returned value from a copy operation now doesn't include org and site components of the paths any more, as these are not in the caller space:

So:

{"copied: [{"src":"bosschaert/my-eds-site/mypath/sub2/foo.html","dst":"bosschaert/my-eds-site/mypath/sub4/foo.html"}]}

is now

{"copied: [{"src":"/mypath/sub2/foo.html","dst":"/mypath/sub4/foo.html"}]}

Related Issues

Contributes to: #169

* Deleting a document moves it to the .trash folder at the root of the
  site storage. If there is already a document with the same name a
  unique suffix is added to the document name.
* Deleting a folder moves all of its content to the .trash under a folder
  with the same name. If a folder with this name already exists in the
  trash a unique suffix is added.

Each document moved to the trash has a new metadata property 'doc-path'
with the path it was originally located at.

Contributes to: #169
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 2, 2026

This PR will trigger a minor release when merged.

@bosschaert bosschaert marked this pull request as ready for review April 2, 2026 15:54
When a file is moved (or copied) with the 'unique' collision option
configured, it will be given a unique name by adding a suffix. This
commit moves that suffix inside the extension rather than appending it,
so for example if the move is done on page1.html to the .trash, and
there is already a file called page1.html in the trash, the newly
trashed document will be called page1-abfsg7h1.html
Comment thread src/source/delete.js Outdated
Comment thread test/source/delete.test.js Outdated
delimiter: '/',
prefix: 'org1/site2/.trash/b/',
})
.reply(200, Buffer.from(BUCKET_LIST_EMPTY_TRASH));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't use you nock.listObjects() ? or nock.listFolder() ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean add listObjects() or listFolder() to the Nock implementation? Because I don't see them there at this time...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to merge main to see them, they were added some time after you started working on your branch.

Copy link
Copy Markdown
Contributor Author

@bosschaert bosschaert Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thanks! I have updated the test to use nock.listObjects() 👍

There are some existing tests that use the old approach. Maybe we should convert them in a seperate PR to avoid this one growing too big...?

Copy link
Copy Markdown
Contributor

@tripodsan tripodsan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have the feeling the code get more and morel over the place. you start exporting common methods in put that are used else where.

maybe it would make sense to separate the source-bus operations from the method handlers. i.e. create a source-bus client.

I would also create a source bus "path" class, that encapsulates all the from/to key,path,name, etc methods.

I also don't like the every growing argument list for the copy operations. maybe this can be consolidated in a combined "CopyOptions" argument (including the "move" flag)

bosschaert and others added 9 commits April 13, 2026 13:43
Co-authored-by: Tobias Bocanegra <tripod@bocanegra.ch>
…lass

Replace individual parameters (src, info, move, opts/fnOpts, collOpts)
with a single CopyOptions object for both copyDocument and copyFolder,
keeping only context as a separate argument.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move getS3Key, getS3KeyFromInfo, and getDocPathFromS3Key out of utils.js
into a dedicated s3-path-utils module. Add tests for all three functions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@bosschaert
Copy link
Copy Markdown
Contributor Author

Hi @tripodsan , thanks for the comments!

i have the feeling the code get more and morel over the place. you start exporting common methods in put that are used else where.

maybe it would make sense to separate the source-bus operations from the method handlers. i.e. create a source-bus client.

I have created source-client.js that now contains common methods dealing directly with the source bus.

I would also create a source bus "path" class, that encapsulates all the from/to key,path,name, etc methods.

Created s3-paths-utils.js with the S3 path conversion functions.

I also don't like the every growing argument list for the copy operations. maybe this can be consolidated in a combined "CopyOptions" argument (including the "move" flag)

I have added a CopyOptions class to source-client.js for this purpose.

@bosschaert bosschaert requested a review from tripodsan April 16, 2026 11:26
Comment thread src/source/delete.js
// Trash a document.
const docName = info.rawPath.split('/').pop();
const srcKey = getS3KeyFromInfo(info);
const newInfo = RequestInfo.clone(info, { path: `/.trash/${docName}` });
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this desired? why don't you include the path somehow? otherwise, deleting common files, like index.html will end up overwriting.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is similar to how a Trash works in an OS environment, e.g. on Mac OS individually deleted files end up in the root of the trash, if there is already a file with that name, they get a unique suffix:

Image

We do a similar thing. So files are not overwritten, but if one with the name already exists, it will get a unique suffix (alpha sortable to get the order in which they are deleted):

Image

Comment thread src/source/source-client.js Outdated
// We do this by appending a ms timestamp to the destination key.

const ext = `.${destKey.split('.').pop()}`;
const destWithoutExt = destKey.slice(0, -ext.length);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this the same as path.basename ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We just want the path to the file, without the extension (so we can insert the unique string). So if we had o1/s1/to.html we want o1/s1/to. With path:

  • path.basename would give me just to.html
  • path.dirname gives me o1/s1
  • path.extname gives me .html
    I guess I could use these to get the same, but it wouldn't be much simpler I think and I would have to import path...

Copy link
Copy Markdown
Contributor

@tripodsan tripodsan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

break up copyWithRetry

* - 'unique' - append a (base-36) encoded timestamp to the destination key to make it unique.
* These timestamps are alphabetically sortable.
*/
async function copyWithRetry(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is very unwieldy. I would break this:

  1. one function that does the retry and one function that does the copy
  2. the copy should be further split in all the collision handling functions

Comment on lines +72 to +76
function getUser(context) {
const email = context.attributes.authInfo?.profile?.email;

return email || 'anonymous';
}
Copy link
Copy Markdown
Contributor

@dominique-pfister dominique-pfister Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no way the authInfo is not set. aren't you looking for:

const { authInfo } = context;
return authInfo.resolveEmail() || 'anonymous';

?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point - I changed this.

Comment thread src/source/delete.js
const opts = { e, log };
opts.status = e.status;
const opts = { e, log: context.log };
opts.status = e.$metadata?.httpStatusCode;
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dominique-pfister since delete now doesn't do bucket.remove() any more but rather a copy, we need to go back to checking e.$metadata?.httpStatusCode am I correct?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants