Commit graph

93 commits

Author SHA1 Message Date
Andre Medeiros
d468a6e88b change how we iterate over accounts 2022-10-31 21:30:12 -04:00
Andre Medeiros
683edbb091 dial down live activities 2022-10-31 20:58:02 -04:00
Andre Medeiros
df0860dd05 increase consumers and make choice more direct 2022-10-31 17:33:41 -04:00
Andre Medeiros
62df908b33 increase maxprocs 2022-10-31 16:09:03 -04:00
Andre Medeiros
3750374aa0 increase maxprocs 2022-10-31 16:04:08 -04:00
Andre Medeiros
00bd99b60e add logging to the scheduler 2022-10-31 15:49:27 -04:00
Andre Medeiros
c9877879da same timezone everywhere 2022-10-27 15:47:26 -04:00
Andre Medeiros
2c31a0df84 increase cron concurrent jobs 2022-10-27 10:09:40 -04:00
Andre Medeiros
030b73e493 increase live activity batch size 2022-10-26 22:55:08 -04:00
Andre Medeiros
c08de574e9 fix context usage in scheduler 2022-10-26 20:48:22 -04:00
Andre Medeiros
13bc85dc8d report on live activities 2022-10-26 20:36:03 -04:00
Andre Medeiros
696f932baa Live activities 2022-10-19 09:37:41 -04:00
Andre Medeiros
01a4ae9559 add pprof endpoints 2022-08-08 12:48:34 -04:00
Andre Medeiros
13af0d1b4a use the right id field 2022-07-31 15:23:29 -04:00
Andre Medeiros
bb2f709259 fix batches 2022-07-31 15:12:31 -04:00
Andre Medeiros
1617bf88e1 less mutations on the accounts table 2022-07-31 15:07:14 -04:00
Andre Medeiros
f3814683b8 don't skimp out on redis connections 2022-07-15 12:55:56 -04:00
Andre Medeiros
f4fea41435 remove some load 2022-07-13 18:43:27 -04:00
Andre Medeiros
63cc67cd86 double pool 2022-07-13 17:45:48 -04:00
Andre Medeiros
fa8b75d728 tone down pool sizes 2022-07-13 17:14:13 -04:00
Andre Medeiros
ae764077b4 increase batch size on scheduler 2022-07-13 17:01:04 -04:00
Andre Medeiros
80bae7ff9f get string slice straight from command result 2022-07-13 16:24:51 -04:00
Andre Medeiros
1f59a47880 increase pool size 2022-07-13 15:52:07 -04:00
Andre Medeiros
be7bd24ae1 increase multiplier 2022-07-13 15:30:39 -04:00
Andre Medeiros
b26140f190 test: less prefetch more workers at the same time 2022-07-13 15:08:21 -04:00
Andre Medeiros
a00e4405c7 tweak redis conns 2022-07-13 15:02:38 -04:00
Andre Medeiros
a3e69954c8 up the batch sizes 2022-07-13 12:39:37 -04:00
Andre Medeiros
2fe34ecc99 run in singleton mode 2022-07-13 12:21:12 -04:00
Andre Medeiros
e80579ed35 tweak batch sizes 2022-07-13 12:19:11 -04:00
André Medeiros
e8a8e5a5b3 chore: use reddit ids when enqueueing accounts (#87) 2022-07-13 11:50:13 -04:00
Andre Medeiros
ce944f92f7 lets prune accounts again 2022-06-30 09:56:38 -04:00
André Medeiros
d3c9e303ff Update api.go 2022-06-27 18:47:07 -04:00
Andre Medeiros
523dce6300 tweak pool 2022-06-27 10:36:01 -04:00
Andre Medeiros
8ae426a4de do not prune accounts 2022-06-25 14:45:05 -04:00
Andre Medeiros
f4e5d7e979 bump connections again 2022-06-25 14:04:03 -04:00
Andre Medeiros
aa92f1ec69 increase API connections 2022-06-25 14:01:03 -04:00
Andre Medeiros
221de1d2b6 back to no singleton mode, now that we're stable 2022-06-04 10:29:33 -04:00
Andre Medeiros
d243cebab6 better iteration 2022-06-04 10:24:22 -04:00
Andre Medeiros
084407a5f2 calculate batch size properly 2022-06-04 10:23:45 -04:00
Andre Medeiros
b23a158471 parallelize scheduler better 2022-06-04 10:21:29 -04:00
Andre Medeiros
6ece4c784a no singleton 2022-05-27 16:12:50 -04:00
Andre Medeiros
0938400dd7 Revert "try enqueue accounts in a loop instead"
This reverts commit 9b5829f549.
2022-05-27 15:02:57 -04:00
Andre Medeiros
01ad9f3d1e Revert "another experiment for scheduling"
This reverts commit c8dd76e1ab.
2022-05-27 15:02:47 -04:00
Andre Medeiros
c8dd76e1ab another experiment for scheduling 2022-05-27 14:55:59 -04:00
Andre Medeiros
9b5829f549 try enqueue accounts in a loop instead 2022-05-27 14:47:59 -04:00
Andre Medeiros
fb3f2d1433 more scheduler tweaks 2022-05-27 14:40:40 -04:00
Andre Medeiros
356402d631 Revert "tweak scheduler"
This reverts commit 38fef3c852.
2022-05-27 14:39:12 -04:00
Andre Medeiros
38fef3c852 tweak scheduler 2022-05-27 14:33:46 -04:00
Andre Medeiros
eb853710c4 Add profiling to scheduler 2022-05-27 13:27:28 -04:00
Andre Medeiros
079cc9a5c5 better variable name 2022-05-23 18:54:38 -04:00