Repository navigation
MSPCA-21 Bulk Notifications - #24
Yurika-Kan wants to merge 14 commits into
Conversation
returns active volunteers matching any of the given foster types, or all active volunteers when the list is empty. pending and inactive are excluded.
- post /notifications emails active volunteers, filtered by foster type, bcc'd in batches of 49 - with sendAt, creates a one-time eventbridge scheduler schedule instead - post /notifications/send-scheduled is called by the dispatch lambda and guarded by a shared secret header - service, controller, guard and module tests
lambda forwards a fired schedule to /notifications/send-scheduled. readme covers the aws resources, env vars, local testing via tunnel, dev test results and the prod checklist.
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
sendNow already drops malformed addresses and dedupes case-insensitively.
…ode-4-Community/mspca into yk/mspca-21-bulk-notifications
coordinators module now imports cognito, so the module spec pulls in jwks-rsa and its esm jose dependency, which jest cannot parse.
teammates only need the env vars to connect. creation steps move below local testing as create from scratch, for prod or a rebuild. notes that the single lambda url is shared.
merge test results into the test table, collapse setup into one script block, trim checklist, troubleshooting and refs.
Juwang110
left a comment
There was a problem hiding this comment.
first pass comments, looks good overall tho!
There was a problem hiding this comment.
okay, wow! this is so well written im amazed yurika! such good documentation, such thorough research done, absolutely wonderful. i havent looked at the tests much since they will likely need updates pending my small changes. id recommend you move everything in the notifs folder aside from the dto, controller, and service (and their respective tests), and put those into a separate aws folder that i recommended. if need be, we can create a new notifications module too if thats necessary, but it shouldnt need much in it.
let me know if you have any confusion, i will thoroughly test and look at each of the tests after the comments are resolved.
also, i have asked max about cleaning up some of our roles within IAM, so be prepared to potentially adjust those within AWS. i wonder now if using terraform may actually be beneficial for us due to how much infrastructure we have setup
| @@ -0,0 +1,38 @@ | |||
| /** | |||
There was a problem hiding this comment.
is there a reason why we put this here instead of in the backend? maybe in an aws/client-scheduler/lambda-functions directory?
there would be a better spot for this if we were using terraform, but we arent, so i think the above suggestion may be better, lmk your thoughts
| // retries (up to 2 by default, set in the function's async invocation config). | ||
| if (!response.ok) { | ||
| throw new Error( | ||
| `Dispatch failed: ${response.status} ${await response.text()}`, |
There was a problem hiding this comment.
is there are way we could potentially include how many retries are left on this?
There was a problem hiding this comment.
lowkey no, lambda doesn't expose attempt # for async retries. workaround is to add a id to the error so we can count the lines with the same id
| return this.repo.find({ | ||
| where: { | ||
| status: VolunteerStatus.ACTIVE, | ||
| ...(fosterTypes.length > 0 && { fosterType: In(fosterTypes) }), |
There was a problem hiding this comment.
Do not Resolve until complete:
note that foster types for each volunteer is going to be made into a list pending the merge of #11, so this will need to change later
| # AWS_NOTIFICATION_DISPATCH_LAMBDA_ARN= | ||
| # The role Scheduler assumes to invoke that Lambda (needs lambda:InvokeFunction): | ||
| # AWS_SCHEDULER_ROLE_ARN= | ||
| # Shared secret the Lambda presents to POST /notifications/send-scheduled. Must match the |
There was a problem hiding this comment.
do we not need to provide the full url that this is to be sent at? for right now, id imagine its localhost:3000/api/notifications/send-scheduled, correct? if so, can we update this?
| ); | ||
| } | ||
|
|
||
| await this.scheduler.send( |
There was a problem hiding this comment.
can this be in a try catch block with some error handling?
| const targetArn = process.env.AWS_NOTIFICATION_DISPATCH_LAMBDA_ARN; | ||
| const roleArn = process.env.AWS_SCHEDULER_ROLE_ARN; | ||
|
|
||
| if (!targetArn || !roleArn) { |
There was a problem hiding this comment.
this shouldnt be needed pending module init changes
| @@ -0,0 +1,130 @@ | |||
| # Scheduled notifications: AWS setup | |||
There was a problem hiding this comment.
i think this README can also go into the new aws folder
| --policy-document "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Action\":\"scheduler:CreateSchedule\",\"Resource\":\"arn:aws:scheduler:${AWS_REGION}:${ACCOUNT_ID}:schedule/default/notification-*\"},{\"Effect\":\"Allow\",\"Action\":\"iam:PassRole\",\"Resource\":\"arn:aws:iam::${ACCOUNT_ID}:role/mspca-notif-scheduler-role\",\"Condition\":{\"StringEquals\":{\"iam:PassedToService\":\"scheduler.amazonaws.com\"}}}]}" | ||
|
|
||
| # 5. backend env: paste into .env or the host's config, then restart | ||
| echo "AWS_NOTIFICATION_DISPATCH_LAMBDA_ARN=arn:aws:lambda:${AWS_REGION}:${ACCOUNT_ID}:function:mspca-notif-dispatch" |
There was a problem hiding this comment.
can we add step by step instructions for where to get each of these env variables here?
| body!: string; | ||
|
|
||
| // Omitted/empty = every active Volunteer regardless of foster type. | ||
| @IsOptional() |
There was a problem hiding this comment.
can we add the validateIf that replaced all of our other IsOptional ones so that we cant put in null here? same with sendAt
match foster types with ArrayOverlap now that fosterType is a list (#11)
… require env vars at boot - move infra/notification-dispatch-lambda to apps/backend/src/aws/scheduler/lambda/notif - fail boot when any notifications env var is missing instead of logging - test findActiveByFosterTypes filtering against array foster types
ℹ️ Issue
Closes MSPCA-21
📝 Description
Foster Coordinators can notify all or a filtered group of volunteers via email sent immediately or at a scheduled time.
POST /notifications(newnotifications/controller, service, module, DTO)subject,body, optionalfilters, optionalsendAt202sent/scheduled ·400bad body ·500SES/Scheduler not configuredsendAtcreates a one time AWS schedule.POST /notifications/send-scheduled, and the schedule deletes itselfPOST /notifications/send-scheduled: Lambda onlyNotificationsDispatchGuard)204sent ·400bad body ·401bad/missing secret@aws-sdk/client-scheduler, new env vars inexample.env, and setup steps ininfra/notification-dispatch-lambda/README.md✔️ Verification
npx jest apps/backend/src/notifications: 43/43 pass (email + Scheduler mocked)filters: ["Dog"]sendAt, bad filter, missing field, line break in subject)Notes
scheduled sends need 3 values in your .env: AWS_NOTIFICATION_DISPATCH_LAMBDA_ARN, AWS_SCHEDULER_ROLE_ARN and NOTIFICATIONS_DISPATCH_SECRET. shared via slackington
emails go to spam: Gmail sender via SES. prod should send from a verified domain (e.g.
c4cneu.com)🏕️ Future Work
@Public()to/send-scheduledCognito guard is global, so the Lambda gets a 401 once auth is on. It works locally only becauseAUTH_DISABLED=true.