| Topic | Presenter | Summary | Duration |
| ------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------- | -------- |
| Progress Bar for the Product Importer | Various Developers | This is the first part of a Friday Dev Training where we worked on adding the Progress Bar for the Product Importer. | 41:50 |00:00:06.970 --> 00:00:10.310 The plan was we were going to do the the
00:00:10.401 --> 00:00:13.575 progress bar for the product importer.
00:00:13.580 --> 00:00:19.989 So. Uhm, yeah. Kind of a.
00:00:22.260 --> 00:00:24.399 Should be fun.
00:00:24.400 --> 00:00:27.270 Uh, so let's pull those notes up.
00:00:27.270 --> 00:00:28.950 Let's let's see what we
00:00:28.950 --> 00:00:31.548 got in there meeting notes.
00:00:31.548 --> 00:00:34.292 There they are. Uhm?
00:00:34.292 --> 00:00:37.188 OK, so we had three things that we
00:00:37.188 --> 00:00:39.947 briefly discussed for implementing it.
00:00:39.950 --> 00:00:42.520 Uh, when we were kind of going through, uh,
00:00:42.520 --> 00:00:45.460 we talked about potentially using signal R.
00:00:45.460 --> 00:00:48.183 Uhm? Uh, we also talked about doing
00:00:48.183 --> 00:00:50.668 pending requests at the back end
00:00:50.668 --> 00:00:52.743 fulfills as updates are available.
00:00:52.750 --> 00:00:55.600 Uhm, we talked about doing a.
00:00:55.600 --> 00:00:58.855 Eric mentioned that I a used status
00:00:58.855 --> 00:01:01.456 pings so the so the front end
00:01:01.456 --> 00:01:03.190 would occasionally ping the back
00:01:03.190 --> 00:01:05.570 end and the back end would respond
00:01:05.645 --> 00:01:07.870 with some updated status progress.
00:01:07.870 --> 00:01:11.440 Uhm? In the front and then update
00:01:11.440 --> 00:01:13.560 the bar accordingly.
00:01:13.560 --> 00:01:15.320 I assume is the way they did that,
00:01:15.320 --> 00:01:20.240 and so yeah, what do you got jail?
00:01:20.240 --> 00:01:22.316 Just a question on requirements there.
00:01:22.320 --> 00:01:24.561 Do we want to be able to refresh the
00:01:24.561 --> 00:01:27.069 page and still see the same progress bar?
00:01:27.070 --> 00:01:28.402 That's a good question.
00:01:28.402 --> 00:01:30.067 Uhm 'cause if we do,
00:01:30.070 --> 00:01:32.282 then simple pings or I guess any
00:01:32.282 --> 00:01:34.715 of them would need to be some
00:01:34.715 --> 00:01:36.410 planted with another set up.
00:01:36.410 --> 00:01:38.930 Or I guess we could have like an
00:01:39.012 --> 00:01:41.844 extra page so be encoded in the URL.
00:01:41.850 --> 00:01:42.900 But either way,
00:01:42.900 --> 00:01:44.300 that's something that would
00:01:44.300 --> 00:01:45.350 require extra work.
00:01:45.350 --> 00:01:46.954 Agreed, UM, and yeah,
00:01:46.954 --> 00:01:49.840 that's a good point because I mean,
00:01:49.840 --> 00:01:51.320 there's a couple ways we could do it.
00:01:51.320 --> 00:01:53.567 Where, uh, I mean one of them.
00:01:53.570 --> 00:01:54.820 We probably would definitely prefer
00:01:54.820 --> 00:01:57.370 not to do, and that's just that.
00:01:57.370 --> 00:02:01.150 If there's any. Uhm?
00:02:01.150 --> 00:02:03.889 If there's any.
00:02:03.890 --> 00:02:07.298 Install or import in progress it
00:02:07.300 --> 00:02:09.008 and it requests progress.
00:02:09.008 --> 00:02:11.143 It sends back progress for.
00:02:11.150 --> 00:02:11.536 That one,
00:02:11.536 --> 00:02:12.887 but then you wouldn't be able to
00:02:12.887 --> 00:02:14.406 have multiple progress bars at once,
00:02:14.410 --> 00:02:16.580 so probably pass on that.
00:02:16.580 --> 00:02:16.885 Uhm,
00:02:16.885 --> 00:02:17.800 would like multiple
00:02:17.800 --> 00:02:19.325 running at the same time.
00:02:21.730 --> 00:02:22.815 You wouldn't want multiple running
00:02:22.815 --> 00:02:25.073 at the same time, you would not,
00:02:25.073 --> 00:02:27.839 because then they could be overlapping.
00:02:27.840 --> 00:02:30.864 Well, what about product imports coming from?
00:02:30.870 --> 00:02:32.980 Multiple stores, like in Ruby,
00:02:32.980 --> 00:02:34.440 you can't really tell them.
00:02:34.440 --> 00:02:36.168 Oh no, you can't do an import right now.
00:02:36.170 --> 00:02:37.210 Somebody else is doing that.
00:02:39.670 --> 00:02:43.606 In theory. The practice would be
00:02:43.606 --> 00:02:45.413 that it would go into the page
00:02:45.413 --> 00:02:46.955 and then the status would be.
00:02:46.960 --> 00:02:48.212 Your import is pending,
00:02:48.212 --> 00:02:50.430 another one is in broad breasted wool,
00:02:50.430 --> 00:02:52.341 and it's it's done will be able
00:02:52.341 --> 00:02:54.050 to start doors kind of thing.
00:02:54.050 --> 00:02:54.364 Theoretically,
00:02:54.364 --> 00:02:56.248 even if it was different stores,
00:02:56.250 --> 00:02:58.455 they could share the same product records.
00:03:00.860 --> 00:03:04.164 Yeah, so we had because of uajy.
00:03:04.170 --> 00:03:07.040 We have a table in the database
00:03:07.040 --> 00:03:08.970 called future product imports.
00:03:08.970 --> 00:03:12.510 Uh, it's under the product schema.
00:03:12.510 --> 00:03:13.450 This future import and
00:03:13.450 --> 00:03:14.390 their future import status.
00:03:14.390 --> 00:03:16.497 The import has a has a record
00:03:16.497 --> 00:03:18.658 of like where the file name is.
00:03:18.660 --> 00:03:20.501 So basically you would load the files
00:03:20.501 --> 00:03:22.746 up onto the like with the file upload.
00:03:22.750 --> 00:03:25.048 Normally it would go into like
00:03:25.048 --> 00:03:27.120 the images products import folder.
00:03:27.120 --> 00:03:28.583 Excuse me and then you would see
00:03:28.583 --> 00:03:30.201 where record in the table for future
00:03:30.201 --> 00:03:33.250 import that was the late to the
00:03:33.250 --> 00:03:34.581 file name and then the timestamp
00:03:34.581 --> 00:03:36.408 of when it should be running the.
00:03:36.410 --> 00:03:38.216 The goal of that feature was that
00:03:38.216 --> 00:03:39.994 they would have vendors who would
00:03:39.994 --> 00:03:41.574 have created upload file that
00:03:41.574 --> 00:03:43.188 would change it like at midnight.
00:03:43.188 --> 00:03:45.420 You know three days or now or whatever.
00:03:45.420 --> 00:03:46.830 Whatever this new launcher product
00:03:46.830 --> 00:03:48.240 is supposed to fill available.
00:03:48.240 --> 00:03:50.096 Uh, I tried to convince them that that
00:03:50.096 --> 00:03:51.835 was probably not the best idea to do it.
00:03:51.840 --> 00:03:53.576 It just instead run the data whatever
00:03:53.576 --> 00:03:55.150 the new products become available,
00:03:55.150 --> 00:03:56.845 but they were adamant about
00:03:56.845 --> 00:03:58.540 how pretty is that way.
00:03:58.540 --> 00:04:01.105 I think what we could do is kind of
00:04:01.105 --> 00:04:02.730 cannibalise a little bit of that.
00:04:02.730 --> 00:04:04.508 Use it here for what I just
00:04:04.508 --> 00:04:05.270 talked about where.
00:04:05.270 --> 00:04:07.614 If let's say that,
00:04:07.614 --> 00:04:08.200 uh,
00:04:08.200 --> 00:04:10.837 no matter who you are you you started import,
00:04:10.840 --> 00:04:13.164 it goes into the first record on
00:04:13.164 --> 00:04:15.210 the feature import table and in
00:04:15.210 --> 00:04:17.166 the run import at date timestamp
00:04:17.166 --> 00:04:21.668 it's just now or whatever and then.
00:04:21.670 --> 00:04:23.539 Part of the import process is that
00:04:23.539 --> 00:04:25.448 it just like runs like scheduled
00:04:25.448 --> 00:04:27.636 task every minute and it just goes
00:04:27.636 --> 00:04:29.340 to the next file that's available
00:04:29.402 --> 00:04:31.280 if one is not already processing.
00:04:31.280 --> 00:04:32.176 A kind of thing,
00:04:32.176 --> 00:04:34.275 and then if it's got a future date
00:04:34.275 --> 00:04:36.644 that's even further out, then so be it.
00:04:36.644 --> 00:04:38.476 It just waits until that future timestamp
00:04:38.476 --> 00:04:40.900 is ready to start the import or whatever,
00:04:40.900 --> 00:04:42.860 and then if it's running now then
00:04:42.860 --> 00:04:46.548 it's going to. Then we add a.
00:04:46.550 --> 00:04:48.956 A status column to that table.
00:04:48.960 --> 00:04:50.238 Right now it doesn't have one.
00:04:50.240 --> 00:04:51.976 We probably just do it on attribute.
00:04:51.980 --> 00:04:54.332 For the moment we put the right of
00:04:54.332 --> 00:04:56.580 schema change and then whenever we
00:04:56.580 --> 00:05:00.390 look at the R on the import page,
00:05:00.390 --> 00:05:02.658 we could get a list back of all the
00:05:02.658 --> 00:05:05.025 pending imports and the status of each one.
00:05:05.030 --> 00:05:06.400 If you're in the admin,
00:05:06.400 --> 00:05:07.730 but if you're in your vendor admin,
00:05:07.730 --> 00:05:08.950 you basically just saying like
00:05:08.950 --> 00:05:10.600 your third in line or whatever
00:05:10.600 --> 00:05:13.414 for for running an import and then
00:05:13.414 --> 00:05:15.608 you know estimated time is law.
00:05:15.610 --> 00:05:16.744 Could figure out some of those metrics.
00:05:16.750 --> 00:05:18.334 Because we will have a sentence
00:05:18.334 --> 00:05:19.888 part that actually tells us how
00:05:19.888 --> 00:05:21.220 fast spreading and how many are
00:05:21.220 --> 00:05:22.608 left per file kind of thing.
00:05:22.610 --> 00:05:23.390 Does that make sense?
00:05:28.410 --> 00:05:33.710 Yes. Comma, however, UM?
00:05:36.510 --> 00:05:37.730 Is that is that specifically
00:05:37.730 --> 00:05:38.950 for what we're doing here,
00:05:38.950 --> 00:05:42.100 or is that to get the like
00:05:42.100 --> 00:05:45.400 staggered imports for athlete?
00:05:45.400 --> 00:05:46.812 Or is that both?
00:05:46.812 --> 00:05:50.075 I guess that that would be to get a
00:05:50.075 --> 00:05:52.250 fullness around stagger to imports for
00:05:52.250 --> 00:05:55.250 the clients who were like Ubi or athlete.
00:05:55.250 --> 00:05:56.834 Should be readily be uploading at
00:05:56.834 --> 00:05:59.338 the same time for touching the same products.
00:05:59.340 --> 00:06:01.620 Uhm, OK.
00:06:01.620 --> 00:06:03.979 Probably a little bit more abstract and
00:06:03.979 --> 00:06:05.972 probably don't need to go quite that
00:06:05.972 --> 00:06:08.548 far in order to get the statuses and values.
00:06:08.550 --> 00:06:10.080 The progress number if we need
00:06:10.080 --> 00:06:11.943 to do it for the page reload
00:06:11.943 --> 00:06:13.770 and be able to still see it,
00:06:13.770 --> 00:06:16.058 we need some kind of state to hold
00:06:16.058 --> 00:06:17.401 onto it beyond memory because
00:06:17.401 --> 00:06:18.943 it's like the apple crashes and
00:06:18.943 --> 00:06:20.680 it reloads like you've lost your
00:06:20.680 --> 00:06:22.105 status and everything you don't.
00:06:22.110 --> 00:06:24.138 It doesn't even know if that
00:06:24.138 --> 00:06:25.840 the file finished or not.
00:06:25.840 --> 00:06:28.170 Kind of thing if we're if we're
00:06:28.170 --> 00:06:29.885 gonna go even more narrow and just
00:06:29.885 --> 00:06:31.824 say it's only valid for as long as
00:06:31.824 --> 00:06:33.559 you're on that product import screen
00:06:33.559 --> 00:06:35.503 after you click the import button.
00:06:35.510 --> 00:06:35.757 Uhm,
00:06:35.757 --> 00:06:37.486 and we're hoping to get a status
00:06:37.486 --> 00:06:39.088 back and it just does alive
00:06:39.088 --> 00:06:40.666 finger and that's all it does.
00:06:40.670 --> 00:06:44.058 That probably did simplest level of effort.
00:06:44.060 --> 00:06:44.885 Just we're making sure that
00:06:44.885 --> 00:06:45.898 everyone had the same page of
00:06:45.898 --> 00:06:46.969 expectation of what it would be like.
00:06:51.930 --> 00:06:54.261 So jail and chat said we could
00:06:54.261 --> 00:06:55.968 potentially have the progress page
00:06:55.968 --> 00:06:58.528 be a separate page in the admin and
00:06:58.596 --> 00:07:01.468 there's an idea in the query param that
00:07:01.468 --> 00:07:02.942 not necessarily completely separate,
00:07:02.942 --> 00:07:05.574 but just playing that you would put
00:07:05.574 --> 00:07:07.502 the job ID in the query parameter.
00:07:07.502 --> 00:07:08.808 So that way, yeah,
00:07:08.808 --> 00:07:10.596 but either way you're just adding
00:07:10.596 --> 00:07:12.454 to the query string that was
00:07:12.454 --> 00:07:13.959 the first thing I thought.
00:07:13.960 --> 00:07:15.976 The second thing I thought as well was
00:07:15.976 --> 00:07:17.693 we could potentially identify and import
00:07:17.693 --> 00:07:20.326 on the back end by the session ID that
00:07:20.326 --> 00:07:22.279 started it so that the person who's.
00:07:22.280 --> 00:07:24.722 Viewing it can still see it, UM,
00:07:24.722 --> 00:07:26.978 if they refresh the page and then we
00:07:26.978 --> 00:07:28.922 would just identify imports on the
00:07:28.922 --> 00:07:31.236 back end by the session that initiated
00:07:31.236 --> 00:07:33.525 them and give them back that one.
00:07:33.530 --> 00:07:36.690 Uhm, if we didn't wanna.
00:07:36.690 --> 00:07:38.730 Uh, do the query program way,
00:07:38.730 --> 00:07:41.257 but I think either way probably is.
00:07:41.260 --> 00:07:43.560 Fine.
00:07:43.560 --> 00:07:45.348 And identifying them by some kind
00:07:45.348 --> 00:07:47.318 of ID would also allow us to.
00:07:47.320 --> 00:07:49.474 I guess share import progress with
00:07:49.474 --> 00:07:51.489 others by giving them the URL.
00:07:51.490 --> 00:07:53.738 The way these session ID for the CART
00:07:53.738 --> 00:07:56.565 works as it gets stored into the session bag,
00:07:56.570 --> 00:07:58.195 which is automatically tide two
00:07:58.195 --> 00:07:59.170 per user session.
00:07:59.170 --> 00:08:00.951 So we could put like the product's
00:08:00.951 --> 00:08:03.477 progress data inside the the users
00:08:03.477 --> 00:08:06.016 session bag in service stack and
00:08:06.016 --> 00:08:07.672 then we'll have no cross pollination
00:08:07.672 --> 00:08:09.544 with that information and it could
00:08:09.544 --> 00:08:12.315 always be queried as long as the that
00:08:12.315 --> 00:08:14.325 key stays inside the session back,
00:08:14.330 --> 00:08:15.119 which is Redis.
00:08:18.810 --> 00:08:19.550 So that could work.
00:08:21.890 --> 00:08:22.690 Cool.
00:08:29.120 --> 00:08:31.668 Well, what are we? What are we?
00:08:31.670 --> 00:08:33.238 What do we think we prefer for it?
00:08:33.240 --> 00:08:36.618 For the for the identification UM?
00:08:36.620 --> 00:08:38.708 I would say probably the less effort way
00:08:38.708 --> 00:08:41.250 would be some kind of ID we put in the
00:08:41.250 --> 00:08:42.913 querystring like jail was saying that
00:08:42.913 --> 00:08:45.977 it would be easier way to do it. Uhm?
00:08:45.977 --> 00:08:50.990 And that idea is to using the session bag.
00:08:50.990 --> 00:08:52.257 We don't even need to worry about
00:08:52.257 --> 00:08:53.520 trying to pull an ID or anything.
00:08:53.520 --> 00:08:56.838 We could just list their import.
00:08:56.840 --> 00:08:59.207 On the page with a simple call that says
00:08:59.207 --> 00:09:01.559 is anything in this session bag for.
00:09:01.560 --> 00:09:02.190 Imports.
00:09:04.800 --> 00:09:07.015 C mean including holding the
00:09:07.015 --> 00:09:09.230 actual like percent progress in
00:09:09.305 --> 00:09:11.210 the session back. Yeah, well.
00:09:14.180 --> 00:09:16.273 Then Howard so so if you update
00:09:16.273 --> 00:09:18.229 the session back on the back end.
00:09:21.120 --> 00:09:23.088 At what point does that information
00:09:23.088 --> 00:09:25.039 get sent back to the user?
00:09:25.040 --> 00:09:27.744 User, I mean we would need to do
00:09:27.744 --> 00:09:31.243 like a front end pinger to to ask for
00:09:31.243 --> 00:09:34.000 a heartbeat of information. I see.
00:09:36.580 --> 00:09:38.464 So like in the Hangfire dashboard, when you
00:09:38.464 --> 00:09:40.390 get that stats call every second. Yeah.
00:09:44.460 --> 00:09:45.776 And then it would be basically asked.
00:09:45.780 --> 00:09:46.920 Is there anything in that
00:09:46.920 --> 00:09:47.832 service bag for this?
00:09:47.840 --> 00:09:50.283 And if yes, then it jumps that
00:09:50.283 --> 00:09:51.880 information onto the screen.
00:09:51.880 --> 00:09:54.292 And you can see it like underneath like the
00:09:54.292 --> 00:09:56.092 normal page is there and the advantages
00:09:56.092 --> 00:09:58.638 to that of of like using session packages,
00:09:58.640 --> 00:09:59.735 its process agnostic.
00:09:59.735 --> 00:10:02.685 So even if it was like behind a
00:10:02.685 --> 00:10:04.939 load balancer or and like you had,
00:10:04.940 --> 00:10:07.005 it was forcibly switching over to a
00:10:07.005 --> 00:10:08.940 different load balancer like all that stuff,
00:10:08.940 --> 00:10:10.578 information could stays in the session back,
00:10:10.580 --> 00:10:13.040 which is across all those things.
00:10:13.040 --> 00:10:14.300 And even if they've logged out
00:10:14.300 --> 00:10:15.811 and log back in or reverse the
00:10:15.811 --> 00:10:17.059 page or any of that stuff,
00:10:17.060 --> 00:10:20.036 it's still part of that stuff.
00:10:20.040 --> 00:10:20.904 Actually no logging out.
00:10:20.904 --> 00:10:22.200 Logging back in would generate discussion,
00:10:22.200 --> 00:10:23.350 so they lose it there, but.
00:10:25.850 --> 00:10:26.250 Yeah.
00:10:28.970 --> 00:10:31.030 So if we so if we want to avoid them,
00:10:31.030 --> 00:10:32.310 you may actually logging out,
00:10:32.310 --> 00:10:34.260 logging back in or like clearing
00:10:34.260 --> 00:10:36.069 their cookies and logging back in,
00:10:36.070 --> 00:10:37.519 where in case we lose the session,
00:10:37.520 --> 00:10:39.527 we would want to use some other kind of
00:10:39.527 --> 00:10:41.538 other identifier that the session itself,
00:10:41.540 --> 00:10:43.439 in which case we could just store to the
00:10:43.439 --> 00:10:45.380 set to the Redis with Seth Cash and just
00:10:45.380 --> 00:10:47.300 give it a key that we we want to use.
00:10:47.300 --> 00:10:49.911 These kinds of things that that makes
00:10:49.911 --> 00:10:52.976 sense maybe by user user ID or something.
00:10:52.980 --> 00:10:54.052 I would say we,
00:10:54.052 --> 00:10:56.580 I mean how big of a deal is it?
00:10:56.580 --> 00:10:58.379 I mean if the back end crashes
00:10:58.379 --> 00:10:59.969 in the middle of an import.
00:10:59.970 --> 00:11:02.920 Or something along those lines.
00:11:02.920 --> 00:11:05.548 I would say that's probably it's
00:11:05.548 --> 00:11:07.818 probably acceptable that product import
00:11:07.818 --> 00:11:10.476 progress doesn't persist in that state.
00:11:10.480 --> 00:11:13.840 I mean, that's I would say that that's.
00:11:13.840 --> 00:11:17.100 We may not need to build it to be that.
00:11:17.100 --> 00:11:20.097 Uh. Robust, I guess.
00:11:20.097 --> 00:11:23.170 UM, something tired job start back over.
00:11:27.050 --> 00:11:30.378 So let's talk about.
00:11:30.378 --> 00:11:31.750 Uhm? In that case.
00:11:31.750 --> 00:11:33.738 So then, let's talk about when
00:11:33.738 --> 00:11:35.773 I actually schedule an import.
00:11:35.780 --> 00:11:37.700 Uh, like I go to the admin dashboard,
00:11:37.700 --> 00:11:40.147 import spreadsheet and hit go uhm?
00:11:40.147 --> 00:11:43.163 What are we? What are we thinking for?
00:11:43.170 --> 00:11:44.802 How we're actually getting
00:11:44.802 --> 00:11:46.842 a status update in general.
00:11:46.850 --> 00:11:49.576 So we talked about like periodic pinging.
00:11:49.576 --> 00:11:52.628 Uhm, which is kind of like the
00:11:52.628 --> 00:11:55.188 status pings that we did for IA,
00:11:55.190 --> 00:11:56.936 uh, we also talked about signal
00:11:56.936 --> 00:11:59.309 R and then the other one was the.
00:11:59.310 --> 00:12:01.837 The front end basically makes a request
00:12:01.837 --> 00:12:04.170 that stays pending until the back end.
00:12:04.170 --> 00:12:07.330 Uh puts something in a queue or something,
00:12:07.330 --> 00:12:08.655 so effectively the front end
00:12:08.655 --> 00:12:10.659 or the back end is is waiting.
00:12:13.090 --> 00:12:15.106 For some trigger and then it responds
00:12:15.106 --> 00:12:17.361 to the request with like a a percentage
00:12:17.361 --> 00:12:19.638 or like a status update and then as soon
00:12:19.638 --> 00:12:21.484 as the front end gets a response back,
00:12:21.484 --> 00:12:23.026 it consumes it updates the status
00:12:23.026 --> 00:12:24.385 bar and then immediately makes
00:12:24.385 --> 00:12:26.247 another request to the back end and
00:12:26.296 --> 00:12:27.997 the back end just holds the request
00:12:27.997 --> 00:12:29.837 until it actually has something to
00:12:29.837 --> 00:12:32.420 say and then and then fulfills it.
00:12:32.420 --> 00:12:34.175 I I've never seen service
00:12:34.175 --> 00:12:35.930 tech operate in that fashion,
00:12:35.930 --> 00:12:37.706 so I don't know how feasible it is.
00:12:37.710 --> 00:12:38.678 The simplest option is
00:12:38.678 --> 00:12:39.646 just one simple heartbeat.
00:12:42.000 --> 00:12:45.248 OK, and jail would you have to say?
00:12:45.250 --> 00:12:47.410 I I'm just talking or think
00:12:47.410 --> 00:12:49.500 about the long running request,
00:12:49.500 --> 00:12:51.677 so I've I've actually started running into
00:12:51.677 --> 00:12:54.339 an issue with trying that way over in Argos.
00:12:54.340 --> 00:12:56.512 Essentially they have this long running
00:12:56.512 --> 00:12:58.562 user imports like thousands of users
00:12:58.562 --> 00:13:00.722 that need to be loaded into the database.
00:13:00.730 --> 00:13:04.168 I've actually found that in my case at least,
00:13:04.170 --> 00:13:05.772 Azure App Services will kill requests
00:13:05.772 --> 00:13:07.819 if you leave them on for long enough,
00:13:07.820 --> 00:13:10.575 so I was just going to say that that
00:13:10.575 --> 00:13:13.665 might be something to consider that
00:13:13.665 --> 00:13:15.530 some it could be an app service.
00:13:15.530 --> 00:13:18.106 Could be some other thing that's sitting
00:13:18.106 --> 00:13:20.646 between Seth and the user might default
00:13:20.646 --> 00:13:23.350 to killing those kinds of long requests.
00:13:23.350 --> 00:13:28.570 So I mean typically fairly be a long request,
00:13:28.570 --> 00:13:31.387 so the the the place that I've seen this,
00:13:31.390 --> 00:13:33.742 this you specifically is this is the
00:13:33.742 --> 00:13:36.441 way that Pubnub listens for messages is
00:13:36.441 --> 00:13:39.330 so Pubnub is like a chat or it's really
00:13:39.410 --> 00:13:42.470 basically just a JSON messaging service.
00:13:42.470 --> 00:13:46.688 And whenever you subscribe to a
00:13:46.688 --> 00:13:50.210 channel in in Pubs API,
00:13:50.210 --> 00:13:53.036 if you look at the dev tools what it
00:13:53.036 --> 00:13:55.779 does is it puts out a request that's
00:13:55.779 --> 00:13:58.678 just it just sends a post request to
00:13:58.678 --> 00:14:00.850 their API and that post requests the
00:14:00.850 --> 00:14:02.810 moment that you receive a message.
00:14:02.810 --> 00:14:05.659 That's when that pending request is filled.
00:14:05.660 --> 00:14:07.868 And after I think it's about 2 minutes.
00:14:07.870 --> 00:14:09.945 If that request hasn't received
00:14:09.945 --> 00:14:11.150 a response it.
00:14:11.150 --> 00:14:13.370 Like the back end automatically fills
00:14:13.370 --> 00:14:15.773 it and basically just says I don't
00:14:15.773 --> 00:14:18.008 have anything for you and then the
00:14:18.008 --> 00:14:19.976 front end creates a new request.
00:14:19.980 --> 00:14:21.266 So again.
00:14:21.266 --> 00:14:25.767 Like a placeholder for the exact page?
00:14:25.770 --> 00:14:26.000 Yeah,
00:14:26.000 --> 00:14:27.610 the front end is sending a request
00:14:27.610 --> 00:14:29.547 to the back end to ask the back end.
00:14:29.550 --> 00:14:29.867 Hey,
00:14:29.867 --> 00:14:32.086 respond to this when you have something
00:14:32.086 --> 00:14:35.528 to tell me and I mean in theory with the
00:14:35.528 --> 00:14:37.328 product import where we're importing,
00:14:37.328 --> 00:14:39.701 you know at least every few seconds
00:14:39.701 --> 00:14:41.769 of product should be going in.
00:14:41.770 --> 00:14:43.543 If it's like a huge product with a ton
00:14:43.543 --> 00:14:45.324 of data and a ton of related stuff,
00:14:45.330 --> 00:14:47.290 but it's relatively simple products,
00:14:47.290 --> 00:14:50.160 I think the speeds like 10 to 15 a second.
00:14:50.160 --> 00:14:51.024 In theory,
00:14:51.024 --> 00:14:53.641 we could be responding every five
00:14:53.641 --> 00:14:56.798 to ten products and not have a
00:14:56.798 --> 00:14:59.501 request staylor or waiting for more
00:14:59.501 --> 00:15:01.530 than like 15 seconds.
00:15:05.320 --> 00:15:07.539 Which you know, a 15 second request.
00:15:07.540 --> 00:15:09.828 While sorry go ahead.
00:15:09.830 --> 00:15:12.500 It would probably be worth.
00:15:12.500 --> 00:15:13.199 Sorry I didn't.
00:15:13.199 --> 00:15:15.130 I didn't hear what you said was that.
00:15:15.130 --> 00:15:16.610 I honestly think that that
00:15:16.610 --> 00:15:18.090 would just be more headache
00:15:18.146 --> 00:15:19.898 than it would probably be worth.
00:15:19.900 --> 00:15:21.965 Trying to set it up and getting
00:15:21.965 --> 00:15:23.660 serviced I could do it and I.
00:15:23.660 --> 00:15:26.186 Trying to enforce that, you know.
00:15:26.190 --> 00:15:27.783 If you had a networking hiccup or any of
00:15:27.783 --> 00:15:29.347 that other stuff was handled correctly,
00:15:29.350 --> 00:15:31.095 they would recreating the request
00:15:31.095 --> 00:15:33.176 when a simple like just asking
00:15:33.176 --> 00:15:34.826 for it every two seconds and
00:15:34.826 --> 00:15:36.819 give us a new status update.
00:15:36.820 --> 00:15:38.330 Whatever the number is currently.
00:15:40.650 --> 00:15:42.738 OK yeah, I mean I'm just that's fine.
00:15:42.740 --> 00:15:46.068 We can do it with the status things.
00:15:46.070 --> 00:15:51.310 Uhm? OK, so we're gonna do.
00:15:51.310 --> 00:15:53.542 Uh, status pings to get the
00:15:53.542 --> 00:15:55.420 progress to the front end.
00:15:55.420 --> 00:15:57.219 How do we identify a request on
00:15:57.219 --> 00:15:59.296 the back end to actually get the
00:15:59.296 --> 00:16:01.156 progress so the request comes in?
00:16:01.160 --> 00:16:04.936 That says what is the import progress on?
00:16:04.940 --> 00:16:08.540 Uh, this import request this,
00:16:08.540 --> 00:16:11.030 you know whatever idea I guess.
00:16:11.030 --> 00:16:13.207 Uhm, if it's not there already then
00:16:13.207 --> 00:16:15.201 it created a new grid. Sendgrid,
00:16:15.201 --> 00:16:18.078 back to the storefront for your client.
00:16:18.080 --> 00:16:19.628 Let them know what the what
00:16:19.628 --> 00:16:20.970 the what to ask for,
00:16:20.970 --> 00:16:22.930 and then we'll use that quit as
00:16:22.930 --> 00:16:24.669 the identifier for the individual.
00:16:24.670 --> 00:16:26.170 It will store that import quick.
00:16:30.550 --> 00:16:32.800 So we will store the import
00:16:32.800 --> 00:16:35.329 good where a say that again.
00:16:35.330 --> 00:16:36.113 I said somewhere,
00:16:36.113 --> 00:16:37.679 but will give me just second.
00:16:37.680 --> 00:16:39.354 Here we see if I what it's what's in
00:16:39.354 --> 00:16:40.986 the current importer right now to.
00:16:40.990 --> 00:16:44.242 Yeah, because we need to modify the
00:16:44.242 --> 00:16:46.702 imported logic itself to actually
00:16:46.702 --> 00:16:49.500 update some progress numbers somewhere.
00:16:49.500 --> 00:16:52.031 And then we'll need to make
00:16:52.031 --> 00:16:54.386 sure that the the debt,
00:16:54.390 --> 00:16:56.465 wherever it's storing that is
00:16:56.465 --> 00:16:58.125 stored with that identifier,
00:16:58.130 --> 00:17:00.642 and then we'll just tie the request to
00:17:00.642 --> 00:17:03.547 take the identifier and grab the progress.
00:17:03.550 --> 00:17:05.582 But the that's the the kind of the
00:17:05.582 --> 00:17:07.765 question at this point is where are we
00:17:07.765 --> 00:17:10.800 storing them and then jail your hand up.
00:17:10.800 --> 00:17:12.021 Yep, so uh.
00:17:12.021 --> 00:17:14.463 Connect also uses progress bars through
00:17:14.463 --> 00:17:17.459 the Hangfire dashboard in the console.
00:17:17.460 --> 00:17:20.316 I don't know that it's feasible yet.
00:17:20.320 --> 00:17:21.835 I'm currently looking at him
00:17:21.835 --> 00:17:23.948 for source code to see if it is.
00:17:23.950 --> 00:17:25.228 Uh, I was wondering if there
00:17:25.228 --> 00:17:27.108 would be a way that we could just
00:17:27.108 --> 00:17:28.566 send them the hangfire job ID.
00:17:28.570 --> 00:17:31.846 So then we the query to the back end
00:17:31.846 --> 00:17:34.998 would ask him fired for the early
00:17:34.998 --> 00:17:37.759 or not operating through hang fire.
00:17:37.760 --> 00:17:40.399 Or we could just make the job
00:17:40.399 --> 00:17:41.530 being fair task.
00:17:41.530 --> 00:17:43.245 I mean, we already have a scheduling
00:17:43.245 --> 00:17:43.980 system with self,
00:17:43.980 --> 00:17:46.430 so he will use it.
00:17:46.430 --> 00:17:47.534 Yeah, that would involve.
00:17:47.534 --> 00:17:49.230 I mean, I guess we could do that,
00:17:49.230 --> 00:17:51.470 but that would involve changing the way
00:17:51.470 --> 00:17:53.827 that we trigger the importer right now,
00:17:53.830 --> 00:17:55.594 which is potentially out of scope
00:17:55.594 --> 00:17:57.390 of what we're doing right now.
00:17:57.390 --> 00:18:00.294 I think what would be different about it?
00:18:00.300 --> 00:18:00.788 I mean,
00:18:00.788 --> 00:18:02.252 right now it's just you send
00:18:02.252 --> 00:18:03.992 to the thing and it runs live
00:18:03.992 --> 00:18:05.132 in memory net right now.
00:18:05.140 --> 00:18:05.424 Yeah,
00:18:05.424 --> 00:18:07.696 yeah you send it and it just starts
00:18:07.696 --> 00:18:09.637 starts crunching and they request.
00:18:09.640 --> 00:18:12.175 Doesn't come back until that
00:18:12.175 --> 00:18:14.710 entire import process is done.
00:18:14.710 --> 00:18:16.474 But we already have to replace
00:18:16.474 --> 00:18:18.120 for the progress bar anyway.
00:18:18.120 --> 00:18:18.577 Correct,
00:18:18.577 --> 00:18:21.776 we need to make the call immediately.
00:18:21.780 --> 00:18:23.649 I think we need to make the
00:18:23.649 --> 00:18:25.802 call immediately return with the
00:18:25.802 --> 00:18:29.858 identifier when you start the import.
00:18:29.860 --> 00:18:32.244 And then so I guess you would have
00:18:32.244 --> 00:18:33.974 to probably either Q or somehow
00:18:33.974 --> 00:18:35.960 if we move it to hang fire.
00:18:35.960 --> 00:18:38.768 We've got to create a hang fire task,
00:18:38.770 --> 00:18:41.083 be able to queue the job instantly from Seth,
00:18:41.090 --> 00:18:43.617 which it doesn't know how to do.
00:18:43.620 --> 00:18:45.676 There's a hang fire has the I think
00:18:45.676 --> 00:18:48.040 it was background job manager class.
00:18:48.040 --> 00:18:49.966 It's just a global static Singleton
00:18:49.966 --> 00:18:51.860 that puts it onto the yeah,
00:18:51.860 --> 00:18:52.959 I know it has all that stuff
00:18:52.959 --> 00:18:53.759 where you could do that,
00:18:53.760 --> 00:18:55.776 but Steph has never actually triggered
00:18:55.776 --> 00:18:57.560 a hangfire job directly before,
00:18:57.560 --> 00:18:59.500 so it would need to learn how to do that.
00:18:59.500 --> 00:19:01.201 But so we need to bring those
00:19:01.201 --> 00:19:02.849 imports in run those extra job
00:19:02.849 --> 00:19:04.577 changes and so that they could.
00:19:04.580 --> 00:19:06.148 Just saying like we could do that,
00:19:06.150 --> 00:19:07.598 it's just going to be that much more
00:19:07.598 --> 00:19:09.275 work to go build all those extra steps.
00:19:09.280 --> 00:19:10.605 I guess what would actually
00:19:10.605 --> 00:19:11.665 be built out though,
00:19:11.670 --> 00:19:15.058 'cause it should just be one import.
00:19:15.060 --> 00:19:16.308 Yes, I'm missing something else there.
00:19:21.830 --> 00:19:23.462 We don't have a way for stuff to
00:19:23.462 --> 00:19:25.227 trigger a job in hang fire right now,
00:19:25.230 --> 00:19:26.820 like to explicitly trigger jobs,
00:19:26.820 --> 00:19:30.100 we have to build out the logic to do that.
00:19:30.100 --> 00:19:32.890 In the scheduler.
00:19:32.890 --> 00:19:34.222 First I would think.
00:19:34.222 --> 00:19:37.253 So we need to have a light API
00:19:37.253 --> 00:19:39.149 to read that information.
00:19:39.150 --> 00:19:41.316 So what you manually create something
00:19:41.316 --> 00:19:43.540 that would talk to it through,
00:19:43.540 --> 00:19:45.780 like adding the the new get package
00:19:45.780 --> 00:19:47.731 for hanging fire two like the
00:19:47.731 --> 00:19:49.887 importer projects and then we have to
00:19:49.950 --> 00:19:52.323 have the Y or something to trigger
00:19:52.323 --> 00:19:54.060 through that process and then we
00:19:54.060 --> 00:19:55.740 have to add the task available in
00:19:55.793 --> 00:19:57.585 hangfire which we do have to add a
00:19:57.585 --> 00:19:59.451 task to the scheduled tasks thing
00:19:59.451 --> 00:20:01.156 that conforms with task capability
00:20:01.156 --> 00:20:03.312 for triggering or process action
00:20:03.312 --> 00:20:06.030 for a task inside hang fire.
00:20:06.030 --> 00:20:08.649 And then we'd have to read hang fire status.
00:20:08.650 --> 00:20:10.768 It updates out of Hank fire,
00:20:10.770 --> 00:20:12.200 it hangfires the one providing
00:20:12.200 --> 00:20:14.210 like a console of like with the
00:20:14.210 --> 00:20:15.760 progress bar is in Seth,
00:20:15.760 --> 00:20:17.685 and then redirect that information
00:20:17.685 --> 00:20:19.760 back to the import page.
00:20:19.760 --> 00:20:20.900 Uh, for all that stuff.
00:20:20.900 --> 00:20:21.950 In order to do all that.
00:20:28.330 --> 00:20:29.534 Jonathan, from your from
00:20:29.534 --> 00:20:31.340 this is what what are we?
00:20:31.340 --> 00:20:34.172 What are we just gaining the use of
00:20:34.172 --> 00:20:36.656 the progress bar by using hang fire?
00:20:36.660 --> 00:20:38.812 A progress bar ends.
00:20:38.812 --> 00:20:41.052 It's basically has at that
00:20:41.052 --> 00:20:42.607 point you have just a.
00:20:42.610 --> 00:20:44.330 I guess more unified API
00:20:44.330 --> 00:20:46.050 for logging inside of that,
00:20:46.050 --> 00:20:47.280 because HANGFIRE already
00:20:47.280 --> 00:20:48.510 provides that's context.
00:20:48.510 --> 00:20:50.610 You can just write to or put
00:20:50.610 --> 00:20:51.750 multiple progress bars on.
00:20:51.750 --> 00:20:52.374 Of course,
00:20:52.374 --> 00:20:54.246 that's predicated on the idea that
00:20:54.246 --> 00:20:56.393 you can just directly read that
00:20:56.393 --> 00:20:58.130 without going through the entire dashboard,
00:20:58.130 --> 00:20:59.684 because we wouldn't be able to go
00:20:59.684 --> 00:21:01.236 through the just plain old hangfire
00:21:01.236 --> 00:21:02.626 dashboard for stuff like movie,
00:21:02.630 --> 00:21:03.915 which I'm still looking at
00:21:03.915 --> 00:21:05.745 the source to see if I could
00:21:05.745 --> 00:21:07.446 figure out how they would do it.
00:21:07.450 --> 00:21:08.126 Of course,
00:21:08.126 --> 00:21:09.816 there's what's James and Brenda
00:21:09.816 --> 00:21:11.170 were talking about with.
00:21:11.170 --> 00:21:14.234 We might need some extra a back end
00:21:14.234 --> 00:21:15.958 infrastructure built out for that.
00:21:22.120 --> 00:21:23.809 Yeah, I'd say.
00:21:23.810 --> 00:21:25.040 I mean from my perspective,
00:21:25.040 --> 00:21:28.676 that sounds that sounds a little.
00:21:28.680 --> 00:21:31.130 A little, adding a framework for one.
00:21:31.130 --> 00:21:32.858 I mean James, the other option
00:21:32.858 --> 00:21:34.640 is what caching on Redis and
00:21:34.640 --> 00:21:36.065 then just like you said,
00:21:36.070 --> 00:21:38.335 providing a grid to read
00:21:38.335 --> 00:21:40.150 that data back. Yeah.
00:21:43.830 --> 00:21:47.900 OK, I mean. I, I think that sounds
00:21:47.900 --> 00:21:49.330 probably the the probably the
00:21:49.388 --> 00:21:51.148 straightforward way to go there.
00:21:51.150 --> 00:21:53.306 UM, and you did. Is that the
00:21:53.306 --> 00:21:55.540 way you did it with I James?
00:21:58.780 --> 00:22:00.875 Ah, that was awhile ago
00:22:00.875 --> 00:22:02.970 and I've slept since then.
00:22:02.970 --> 00:22:06.083 Uh, Are you sure? I can't remember.
00:22:06.083 --> 00:22:06.807 Yeah, I can't remember.
00:22:06.810 --> 00:22:08.178 I just know there were status pins on it,
00:22:08.180 --> 00:22:09.780 so that's why I brought it up that
00:22:09.780 --> 00:22:13.364 once happened, but. OK, so do we.
00:22:13.364 --> 00:22:14.870 Is that our process moving forward?
00:22:14.870 --> 00:22:16.798 Then I just wanted.
00:22:16.800 --> 00:22:17.528 See where we're going?
00:22:22.540 --> 00:22:24.196 And burn in line. I think you're you're
00:22:24.196 --> 00:22:25.907 muted if you're talking if you're not.
00:22:28.180 --> 00:22:31.600 Talk. Uh, I was muted,
00:22:31.600 --> 00:22:33.300 but I was not talking.
00:22:33.300 --> 00:22:39.034 I was a. Thinking. As one does.
00:22:39.034 --> 00:22:41.302 Uh, I'm reviewing the code and
00:22:41.302 --> 00:22:43.405 currently the the import products
00:22:43.405 --> 00:22:45.883 from Excel endpoints does not like
00:22:45.964 --> 00:22:48.663 generate an identifier or anything it,
00:22:48.663 --> 00:22:50.028 but it's mostly just using
00:22:50.028 --> 00:22:53.870 it off the file name. Uhm?
00:22:53.870 --> 00:22:55.814 Which should be fine because the
00:22:55.814 --> 00:22:57.110 filename automatically iterates as
00:22:57.158 --> 00:22:59.013 it uploads to the server and that
00:22:59.013 --> 00:23:00.117 iterated filename got returned
00:23:00.117 --> 00:23:01.874 to the client and then they when
00:23:01.874 --> 00:23:03.358 the import button was pressed,
00:23:03.358 --> 00:23:05.640 the iterated name is what gets sent
00:23:05.708 --> 00:23:07.610 into the import parts of itself.
00:23:07.610 --> 00:23:11.530 Uh, so in theory the the file name is enough,
00:23:11.530 --> 00:23:13.077 though later on we might want to
00:23:13.077 --> 00:23:14.432 do something more robust with like
00:23:14.432 --> 00:23:15.944 a quid or something like that to
00:23:15.993 --> 00:23:18.466 help identify that the same job or a
00:23:18.466 --> 00:23:20.695 separate job running against the same
00:23:20.695 --> 00:23:23.460 filename kind of thing on the server.
00:23:23.460 --> 00:23:24.252 But that's OK,
00:23:24.252 --> 00:23:26.100 that's that might be still Creek there.
00:23:30.350 --> 00:23:33.199 OK, so you're saying so whenever you
00:23:33.199 --> 00:23:35.639 actually upload the file that does the
00:23:35.639 --> 00:23:37.959 upload and when the upload is done the
00:23:37.959 --> 00:23:39.898 back end responds and says this is
00:23:39.898 --> 00:23:42.369 the file name that I have and then the
00:23:42.369 --> 00:23:44.410 import runs off of that file name too.
00:23:44.410 --> 00:23:46.634 So we already have a piece of information
00:23:46.634 --> 00:23:49.162 that both the front and back and have to
00:23:49.162 --> 00:23:51.018 sync like basically to synchronize on.
00:23:51.020 --> 00:23:52.735 So then we would just need to
00:23:52.735 --> 00:23:54.723 actually build the back end logic that
00:23:54.723 --> 00:23:56.223 updates a progress bar somewhere,
00:23:56.230 --> 00:23:58.260 updates a percentage of progress
00:23:58.260 --> 00:23:59.740 somewhere and then the end point
00:23:59.740 --> 00:24:01.150 for the front for the front.
00:24:01.150 --> 00:24:03.040 And to request that progress.
00:24:05.110 --> 00:24:05.650 Yes.
00:24:07.780 --> 00:24:09.852 OK cool, so how do we let's talk
00:24:09.852 --> 00:24:11.853 about that then where do we want
00:24:11.853 --> 00:24:13.700 to store the the progress state?
00:24:13.700 --> 00:24:20.387 UM, in the. Uh, in the back end and.
00:24:20.390 --> 00:24:22.046 How do we want to send it out?
00:24:22.050 --> 00:24:23.960 Or how do we wanna read it out I guess?
00:24:26.220 --> 00:24:29.160 And I'm going to open up my.
00:24:29.160 --> 00:24:32.640 Uh, we wanna do this against
00:24:32.640 --> 00:24:34.228 release 2021.1 or develop.
00:24:41.900 --> 00:24:44.780 Ah, I've got it open at
00:24:44.780 --> 00:24:45.988 what would be developed.
00:24:49.370 --> 00:24:50.570 Doing my PR ever goes there?
00:25:02.100 --> 00:25:03.999 OK, uhm. Alright,
00:25:03.999 --> 00:25:09.559 so so James do you wanna do the uh?
00:25:09.560 --> 00:25:11.387 The screen or do you want me
00:25:11.387 --> 00:25:13.008 to share and and kind of?
00:25:13.010 --> 00:25:15.068 I'd say I'd say, let's uhm.
00:25:17.280 --> 00:25:18.852 I'd say we could maybe do
00:25:18.852 --> 00:25:20.140 it on somebody else is.
00:25:20.140 --> 00:25:21.778 But yeah, maybe Brendan if you
00:25:21.778 --> 00:25:23.790 wanna share and then, uh, OK,
00:25:23.790 --> 00:25:26.436 does anybody else or anybody else
00:25:26.436 --> 00:25:28.451 interested in kind of taking
00:25:28.451 --> 00:25:30.819 the taking the point on it or?
00:25:30.820 --> 00:25:32.815 Yeah, I'd love if somebody
00:25:32.815 --> 00:25:34.810 would would a volunteer here.
00:25:34.810 --> 00:25:35.670 I've got it open.
00:25:38.260 --> 00:25:39.890 Why not James? But you know how to do this.
00:25:53.590 --> 00:25:56.206 Brandon, do you wanna volentold somebody?
00:25:56.210 --> 00:25:57.234 I mean, Brendan Flaherty.
00:25:57.234 --> 00:25:59.349 This was your this was your love child,
00:25:59.350 --> 00:26:01.060 so once you once you get in there man.
00:26:04.110 --> 00:26:09.200 I might just open it up. Uhm?
00:26:09.200 --> 00:26:10.418 I do it in the release build,
00:26:10.420 --> 00:26:11.059 was that correct?
00:26:13.250 --> 00:26:15.194 For the race release branch, yeah,
00:26:15.194 --> 00:26:16.598 we're gonna we're gonna do this.
00:26:16.600 --> 00:26:17.671 Yeah I was I was thinking it's
00:26:17.671 --> 00:26:18.698 really 'cause I think right now,
00:26:18.700 --> 00:26:19.900 developed at least earlier this
00:26:19.900 --> 00:26:21.398 week I was able actually get
00:26:21.398 --> 00:26:22.464 developed to build. I don't?
00:26:22.464 --> 00:26:23.790 I assume that's probably fixed by now, but.
00:26:26.280 --> 00:26:29.656 OK, give me a few minutes and I'll
00:26:29.660 --> 00:26:32.160 pull down your solution. Awesome.
00:26:34.260 --> 00:26:35.924 Sorry James I know you just you just
00:26:35.924 --> 00:26:38.096 know how to do this, so trying to go
00:26:38.096 --> 00:26:39.890 more by committee here in training.
00:26:48.330 --> 00:26:50.470 Does somebody wanna like pseudocode?
00:26:50.470 --> 00:26:52.478 Some of that out. While he's doing this.
00:26:55.140 --> 00:26:56.850 I would say the only thing left we have
00:26:56.850 --> 00:26:58.745 to really discuss 'cause this should
00:26:58.745 --> 00:27:00.400 be relatively simple in implementation.
00:27:00.400 --> 00:27:03.664 The only thing we have to discuss next
00:27:03.664 --> 00:27:06.376 is we have to hook into somewhere in
00:27:06.376 --> 00:27:08.777 the importer to basically calculate
00:27:08.777 --> 00:27:11.672 like products imported divided by
00:27:11.672 --> 00:27:14.334 total products in the sheet or I
00:27:14.334 --> 00:27:15.419 guess products integrated and that
00:27:15.419 --> 00:27:16.780 would be our progress number.
00:27:16.780 --> 00:27:18.404 So we need to find where we're going
00:27:18.404 --> 00:27:20.555 to do that and then the last thing
00:27:20.555 --> 00:27:22.438 is again figuring out where we're
00:27:22.438 --> 00:27:24.400 going to actually store the data.
00:27:24.400 --> 00:27:26.486 Uhm, so that we can respond to
00:27:26.486 --> 00:27:28.068 the friends request. So uhm.
00:27:28.068 --> 00:27:30.476 I think the organ I don't know
00:27:30.476 --> 00:27:32.298 who the organizer is.
00:27:32.300 --> 00:27:32.932 This meeting,
00:27:32.932 --> 00:27:34.828 that's probably think it's just organization.
00:27:34.830 --> 00:27:37.380 But can we bring that?
00:27:37.380 --> 00:27:38.080 If you haven't already,
00:27:38.080 --> 00:27:39.130 or James if you haven't already?
00:27:39.130 --> 00:27:40.470 Can somebody write down everything
00:27:40.470 --> 00:27:42.132 that we so far have talked
00:27:42.132 --> 00:27:43.457 about needing to do here?
00:27:45.540 --> 00:27:49.872 I got it. As in other words,
00:27:49.872 --> 00:27:51.440 like there's if you're an organizer,
00:27:51.440 --> 00:27:53.669 then you have access to like meeting notes
00:27:53.670 --> 00:27:55.700 and like the 33 AM,
00:27:55.700 --> 00:27:58.140 so I was able to get to the meeting notes
00:27:58.140 --> 00:27:59.740 You have to like go to your calendar,
00:27:59.740 --> 00:28:01.119 open the meeting, and then there's a
00:28:01.119 --> 00:28:02.936 tab at the top that says meeting notes
00:28:02.940 --> 00:28:06.262 a kind of a weird clunky way.
00:28:06.262 --> 00:28:08.008 And then once somebody creates it,
00:28:08.010 --> 00:28:09.855 everyone can see it on the three dot menu,
00:28:09.860 --> 00:28:11.076 right? I think so.
00:28:11.076 --> 00:28:12.900 I already created some for this,
00:28:12.900 --> 00:28:16.329 so I don't see them in here either though.
00:28:16.330 --> 00:28:18.046 I don't got a meeting info.
00:28:18.050 --> 00:28:18.682 Yeah, not in there.
00:28:18.682 --> 00:28:19.630 I think you have to get
00:28:19.668 --> 00:28:20.548 to it from the calendar,
00:28:20.550 --> 00:28:22.400 which is kind of goofy, OK?
00:28:25.350 --> 00:28:26.418 Alright, let's see.
00:28:26.418 --> 00:28:29.630 Uh oh and David to answer your question.
00:28:29.630 --> 00:28:32.114 Uh, we do not have a way to immediately
00:28:32.114 --> 00:28:33.569 trigger an email and stuff.
00:28:33.570 --> 00:28:35.724 I don't believe we have these
00:28:35.724 --> 00:28:37.160 scheduled tasks that sends
00:28:37.226 --> 00:28:38.830 them when they're queued,
00:28:38.830 --> 00:28:40.096 and that's relative.
00:28:40.096 --> 00:28:41.784 There is a button,
00:28:41.790 --> 00:28:43.005 but it does not interact
00:28:43.005 --> 00:28:44.220 with hangfire to do that.
00:28:46.940 --> 00:28:48.727 It goes directly to the like, uh,
00:28:48.727 --> 00:28:51.156 it goes directly to a thing that
00:28:51.156 --> 00:28:53.464 that reads that class that was
00:28:53.464 --> 00:28:55.449 that hangfire is also reading,
00:28:55.450 --> 00:28:57.370 but it's not scheduling a job
00:28:57.370 --> 00:28:58.890 and hang fire triggering me.
00:28:58.890 --> 00:29:00.750 Gotcha, OK, thank you.
00:29:27.700 --> 00:29:29.010 I tabbed off of it, but.
00:29:31.400 --> 00:29:32.709 Put the meeting notes
00:29:32.709 --> 00:29:34.128 just kind of clarify what you're putting
00:29:34.128 --> 00:29:37.444 as you put Em, Brandon. Yeah, sure.
00:29:37.444 --> 00:29:39.660 So on the first bullet point I have,
00:29:39.660 --> 00:29:41.946 how do we sync the state of the import
00:29:41.946 --> 00:29:44.066 between front end and back end and the
00:29:44.066 --> 00:29:46.450 sort of answer to that question is when
00:29:46.517 --> 00:29:48.925 the sheet is uploaded to the back end,
00:29:48.930 --> 00:29:51.138 the response contains the file name.
00:29:51.140 --> 00:29:53.844 The back end will use for the import,
00:29:53.850 --> 00:29:55.194 so this gives both front and
00:29:55.194 --> 00:29:56.090 back end of value.
00:29:56.090 --> 00:29:58.547 They both know to use to sync or the
00:29:58.547 --> 00:30:01.166 front end can use to request that update.
00:30:01.170 --> 00:30:04.190 And the next one is how are we storing UM,
00:30:04.190 --> 00:30:05.062 progress information?
00:30:05.062 --> 00:30:09.009 I think what we're kind of sounds like we're
00:30:09.009 --> 00:30:12.105 leaning toward is storing it in the cache.
00:30:12.110 --> 00:30:13.910 Uhm?
00:30:13.910 --> 00:30:17.005 So we can use the stuff cash and have the
00:30:17.005 --> 00:30:19.124 key part of the key be that file name
00:30:19.124 --> 00:30:21.077 and then store the progress there and
00:30:21.077 --> 00:30:23.300 then when the front end makes a request,
00:30:23.300 --> 00:30:25.337 we just read that cash value out
00:30:25.337 --> 00:30:28.870 and send it over the wire. Uhm?
00:30:28.870 --> 00:30:32.065 So I think that's how we'll do it so.
00:30:32.070 --> 00:30:38.728 Cash. To store the progress.
00:30:38.730 --> 00:30:41.076 Using the import file name as
00:30:41.076 --> 00:30:42.640 part of the key.
00:30:46.630 --> 00:30:50.585 Reading the value out to fulfill requests.
00:30:53.880 --> 00:30:57.200 Uhm, and then the last sort of question.
00:30:57.200 --> 00:30:59.883 I guess that we have is, uhm,
00:30:59.883 --> 00:31:02.564 what metric are we using to update
00:31:02.564 --> 00:31:05.084 the percentage and I think that's
00:31:05.084 --> 00:31:07.179 just number of products imported
00:31:07.179 --> 00:31:09.656 divided by total number of products.
00:31:09.660 --> 00:31:11.568 On this on the spreadsheet so.
00:31:14.130 --> 00:31:20.010 And if we can additionally include. Uhm?
00:31:20.010 --> 00:31:22.796 Yeah, just like a progress decimal numbers.
00:31:22.800 --> 00:31:25.248 I think we can put a couple of
00:31:25.248 --> 00:31:26.888 different additional numbers of like.
00:31:26.890 --> 00:31:29.130 Like how long you know what's the
00:31:29.130 --> 00:31:31.439 frequency of the products so and an
00:31:31.439 --> 00:31:33.939 estimate of how long it's gonna take to.
00:31:33.940 --> 00:31:36.354 Finish based upon the current free. Yeah,
00:31:36.354 --> 00:31:39.540 so we could probably have most of that be.
00:31:39.540 --> 00:31:43.516 Uhm? Be handled on the front end.
00:31:43.520 --> 00:31:45.476 If we respond with total products
00:31:45.476 --> 00:31:47.533 that like past the import and
00:31:47.533 --> 00:31:49.238 are on the integrate phase,
00:31:49.240 --> 00:31:51.000 so that's like the total.
00:31:51.000 --> 00:31:53.868 The total number that are actually
00:31:53.868 --> 00:31:55.302 good importable products.
00:31:55.310 --> 00:31:57.526 Uhm, divided and so we send that number
00:31:57.526 --> 00:31:59.905 back in the response to the front end.
00:31:59.910 --> 00:32:03.755 The number done so far and I guess
00:32:03.755 --> 00:32:05.630 time elapsed since it began.
00:32:05.630 --> 00:32:07.538 The front end can calculate all
00:32:07.538 --> 00:32:09.567 those things without us having to do
00:32:09.567 --> 00:32:10.953 it on the back end and.
00:32:10.960 --> 00:32:12.724 Uh, and update more information we need.
00:32:12.730 --> 00:32:16.050 So then in this the cash would contain
00:32:16.050 --> 00:32:19.254 a start time and A and then a separate
00:32:19.254 --> 00:32:22.457 piece for total number of products and
00:32:22.457 --> 00:32:25.918 products complete in the front end can do a.
00:32:25.920 --> 00:32:27.747 Uh, you know it's been running for
00:32:27.747 --> 00:32:30.380 10 seconds and 40 products are done,
00:32:30.380 --> 00:32:32.408 so we're doing 4 per second
00:32:32.410 --> 00:32:34.102 and there's 100 products.
00:32:34.102 --> 00:32:36.640 So that means there's whatever second
00:32:36.709 --> 00:32:39.053 and a half left or 15 seconds left.
00:32:39.060 --> 00:32:43.886 Uhm so. Uhm? Yeah, I would say that's a.
00:32:46.070 --> 00:32:47.054 That probably is good.
00:32:47.054 --> 00:32:50.280 We should be able to, uh.
00:32:50.280 --> 00:32:52.390 Derive everything we need from
00:32:52.390 --> 00:32:56.669 distorting that so progress metric is.
00:32:56.669 --> 00:32:59.678 Total products integrated.
00:32:59.680 --> 00:33:01.316 Divided by.
00:33:01.316 --> 00:33:06.224 Total products that will be integrated.
00:33:06.230 --> 00:33:08.435 Also respond with.
00:33:08.435 --> 00:33:14.780 The start time of the import. So that.
00:33:14.780 --> 00:33:19.239 If you can calculate products per second.
00:33:19.240 --> 00:33:21.442 And an estimated.
00:33:21.442 --> 00:33:22.910 Completion time.
00:33:27.320 --> 00:33:28.990 OK, I think that, uh.
00:33:32.060 --> 00:33:33.350 That sounds, uh.
00:33:35.480 --> 00:33:36.170 Pretty good.
00:33:40.410 --> 00:33:44.938 I'm gonna go ahead and hit a. So not F11.
00:33:44.938 --> 00:33:48.840 Oh how I even did that. Uh, enter.
00:33:48.840 --> 00:33:51.836 Actually it's the key was gonna press.
00:33:51.840 --> 00:33:55.921 Uhm? Cool, so I think that pretty
00:33:55.921 --> 00:33:58.960 much covers all of the stuff,
00:33:58.960 --> 00:34:01.660 so I think we're ready to.
00:34:01.660 --> 00:34:03.910 Write some code also, thanks.
00:34:09.370 --> 00:34:12.208 So, uh, Brendan, were you gonna?
00:34:12.210 --> 00:34:13.418 Steal the screen and.
00:34:15.740 --> 00:34:20.960 For or was I like. Hearing things.
00:34:38.720 --> 00:34:44.270 Cool. So let's say, uhm.
00:34:44.270 --> 00:34:47.542 The first thing we probably want to do
00:34:47.542 --> 00:34:52.810 is in the actual import process, uhm?
00:34:52.810 --> 00:34:56.306 We want to go into the integrate products
00:34:56.306 --> 00:34:58.300 method and somewhere in there it's
00:34:58.300 --> 00:35:00.430 going to be iterating over products.
00:35:00.430 --> 00:35:03.200 It's got to be. Uhm?
00:35:06.640 --> 00:35:08.560 So we'll look into that,
00:35:08.560 --> 00:35:10.954 and each time a product is done,
00:35:10.960 --> 00:35:13.978 we'll update the cache with the,
00:35:13.980 --> 00:35:15.456 uh, the updated information.
00:35:15.456 --> 00:35:18.304 Or we could potentially maybe not do it
00:35:18.304 --> 00:35:20.292 every single time or product is done.
00:35:20.300 --> 00:35:22.524 Maybe do it like every five products or
00:35:22.524 --> 00:35:26.080 ten products or something. Technically.
00:35:26.080 --> 00:35:29.440 There are two separate phases.
00:35:29.440 --> 00:35:31.870 And maybe we want to do both of them
00:35:31.870 --> 00:35:33.520 together as like the first 50%.
00:35:33.520 --> 00:35:34.835 It's based learning the 2nd
00:35:34.835 --> 00:35:36.219 50% it's phase two.
00:35:36.219 --> 00:35:37.338 And the first.
00:35:37.340 --> 00:35:39.440 So the first phase would be every
00:35:39.440 --> 00:35:41.897 time it table read our record out of
00:35:41.897 --> 00:35:43.933 like the Excel file and then that
00:35:43.933 --> 00:35:46.150 gives you your counts as it goes up.
00:35:46.150 --> 00:35:47.476 It basically just says like part
00:35:47.476 --> 00:35:48.910 of the report is discovered.
00:35:48.910 --> 00:35:50.678 You know 53 items and if they know
00:35:50.678 --> 00:35:52.789 they're in their spreadsheet they have 1000.
00:35:52.790 --> 00:35:53.726 Then they kind of, you know,
00:35:53.730 --> 00:35:55.557 and all the cover had how many
00:35:55.557 --> 00:35:57.364 rows there going to roughly be
00:35:57.364 --> 00:35:59.054 reading through and then the once
00:35:59.054 --> 00:36:00.362 it hits the 50% mark.
00:36:00.362 --> 00:36:01.672 That means that it's finished
00:36:01.672 --> 00:36:03.581 reading all the data out of Excel
00:36:03.581 --> 00:36:05.207 and now it's gonna start processing
00:36:05.207 --> 00:36:06.698 important events that we could even.
00:36:06.698 --> 00:36:06.920 Actually,
00:36:06.920 --> 00:36:08.865 potentially make this 22 progress
00:36:08.865 --> 00:36:11.553 bars so the first progress bar you
00:36:11.553 --> 00:36:13.575 know the status is discovering and
00:36:13.575 --> 00:36:15.763 once that progress bar it's 100
00:36:15.763 --> 00:36:17.620 status changes to integrating and
00:36:17.620 --> 00:36:20.315 then the progress goes back to zero.
00:36:20.320 --> 00:36:22.635 And also another thing I'm
00:36:22.635 --> 00:36:24.024 thinking about here.
00:36:24.030 --> 00:36:26.262 Uhm, by reading and writing out of the cash,
00:36:26.270 --> 00:36:28.412 we actually give this the ability
00:36:28.412 --> 00:36:30.625 to potentially cancel and import by
00:36:30.625 --> 00:36:33.110 giving the front end and end point,
00:36:33.110 --> 00:36:35.140 they can hit two cancel and it
00:36:35.140 --> 00:36:37.740 adds a flag to that cash and then
00:36:37.740 --> 00:36:40.230 before each new item gets imported,
00:36:40.230 --> 00:36:42.039 it checks the cash to see if a flag
00:36:42.039 --> 00:36:43.823 is set on that key and if it is,
00:36:43.830 --> 00:36:45.642 it cuts, you know,
00:36:45.642 --> 00:36:48.328 breaks the loop and updates the
00:36:48.328 --> 00:36:50.704 status of the. Import to cancelled.
00:36:50.704 --> 00:36:51.062 Uhm?
00:36:51.062 --> 00:36:54.018 And then the front end you know on
00:36:54.018 --> 00:36:56.208 its next status request reads that
00:36:56.208 --> 00:36:58.395 the status is cancelled and can
00:36:58.395 --> 00:37:00.375 show you Noah. Cancelled thing uhm.
00:37:00.375 --> 00:37:02.580 I am very for that considering it
00:37:02.643 --> 00:37:04.561 will add very little scope but will
00:37:04.561 --> 00:37:06.566 add a large benefit to the people
00:37:06.566 --> 00:37:09.295 who have to like fix their imports.
00:37:09.295 --> 00:37:12.430 It's true Yep.
00:37:12.430 --> 00:37:14.776 Cool well that seems like a
00:37:14.776 --> 00:37:17.620 pretty pretty easy win there so.
00:37:17.620 --> 00:37:20.090 Uhm?
00:37:20.090 --> 00:37:22.876 So somewhere in here, uhm, OK,
00:37:22.876 --> 00:37:24.060 I integrate product sizing,
00:37:24.060 --> 00:37:24.930 so that's what I was.
00:37:24.930 --> 00:37:27.090 That's what I was thinking earlier.
00:37:27.090 --> 00:37:30.177 So I think load async spreadsheet model.
00:37:30.180 --> 00:37:33.468 I believe that's where it actually.
00:37:33.470 --> 00:37:35.528 That I think that's where it loads
00:37:35.528 --> 00:37:36.981 the spreadsheet and then parse
00:37:36.981 --> 00:37:38.773 async is these sort of first Phase
00:37:38.773 --> 00:37:40.767 I think that that JJ was talking
00:37:40.767 --> 00:37:45.070 about it parses the spreadsheet and.
00:37:45.070 --> 00:37:48.878 And, uh, basically figures out how many.
00:37:48.880 --> 00:37:50.926 Products there are and breaks them
00:37:50.926 --> 00:37:53.449 into the import items and all that jazz.
00:38:04.540 --> 00:38:09.170 And that person is online 52. Yeah,
00:38:09.170 --> 00:38:12.370 so there's the load the Parson the resolve.
00:38:12.370 --> 00:38:15.934 Yeah, and I think resolve is the one that's
00:38:15.934 --> 00:38:19.908 actually doing the storing into the database.
00:38:19.910 --> 00:38:22.927 Load load reads the spreadsheet into memory,
00:38:22.930 --> 00:38:26.210 parse parses it into import items that the
00:38:26.210 --> 00:38:29.350 resolve step uses to create product models,
00:38:29.350 --> 00:38:31.374 or update product models.
00:38:31.374 --> 00:38:34.214 Brendan, what you're looking for is a
00:38:34.214 --> 00:38:36.960 registry loader wrapper dot get cash client.
00:38:42.920 --> 00:38:44.790 Rapper Loader rapper I think.
00:38:48.310 --> 00:38:49.150 That oughta do it.
00:38:51.460 --> 00:38:53.266 You have to pass in anything
00:38:53.266 --> 00:38:55.260 on that or is it just a?
00:38:55.260 --> 00:38:56.980 I think context profile name.
00:39:09.060 --> 00:39:12.580 It could be a sink, but I'm not sure. I had
00:39:19.960 --> 00:39:20.668 Yep, there it is.
00:39:28.780 --> 00:39:30.665 Alright, so that's gonna give
00:39:30.665 --> 00:39:33.380 us the cash client so. Uhm?
00:39:35.870 --> 00:39:37.490 Cool, so then what we could
00:39:37.490 --> 00:39:39.239 do is I would say yeah.
00:39:39.240 --> 00:39:41.088 So the very first thing that
00:39:41.088 --> 00:39:42.852 needs to happen is. Uhm?
00:39:42.852 --> 00:39:44.980 It's probably like file.
00:39:44.980 --> 00:39:49.393 Is it filename or? Maybe on that
00:39:49.393 --> 00:39:51.698 model dot filename or something?
00:39:51.700 --> 00:39:54.538 Gotta get gotta get from somewhere.
00:39:54.540 --> 00:39:56.364 Unless that spreadsheet model
00:39:56.364 --> 00:39:58.644 is just already contains the
00:39:58.644 --> 00:40:00.998 stream that will read it in from.
00:40:01.000 --> 00:40:02.228 I guess it does.
00:40:02.228 --> 00:40:04.070 So how does that get created?
00:40:04.070 --> 00:40:04.622 Yeah, well,
00:40:04.622 --> 00:40:06.554 let's let's go over to the end
00:40:06.554 --> 00:40:08.099 point and start from there.
00:40:08.100 --> 00:40:09.675 'cause that's going to be more relevant.
00:40:09.680 --> 00:40:11.015 Information of like which I
00:40:11.015 --> 00:40:12.083 think it's actually calling.
00:40:21.450 --> 00:40:22.997 A control F-12 if you if you
00:40:22.997 --> 00:40:24.738 go back to where you were in
00:40:24.738 --> 00:40:26.268 control of 12 on that
00:40:26.270 --> 00:40:27.702 it takes to implementation.
00:40:27.702 --> 00:40:30.708 I am a fool. You can right click
00:40:30.708 --> 00:40:32.668 and do find all references.
00:40:32.670 --> 00:40:33.610 Do you have Resharper?
00:40:33.610 --> 00:40:35.370 It's shift at 12 to find music.
00:40:37.670 --> 00:40:39.842 Shift at 12 is default for
00:40:39.842 --> 00:40:41.290 Visual Studio for references.
00:40:48.680 --> 00:40:50.227 Seems like it's only finding the one,
00:40:50.230 --> 00:40:51.910 but this is obviously being called somewhere.
00:40:56.480 --> 00:40:57.860 Is it part of the interface?
00:40:57.860 --> 00:40:59.603 Can we find the interface of the
00:40:59.603 --> 00:41:01.458 class and do references on that?
00:41:01.460 --> 00:41:03.539 Yeah, that's, but I think it's just
00:41:03.539 --> 00:41:05.800 gonna take you to the implementation.
00:41:05.800 --> 00:41:08.624 Uh, the the endpoint is in product import
00:41:08.624 --> 00:41:11.128 service dot CS in the service project.
00:41:28.680 --> 00:41:28.970 Oh
00:41:40.020 --> 00:41:42.996 you gotta restore Nougat package is.
00:41:43.000 --> 00:41:44.820 It's probably all those errors
00:41:44.820 --> 00:41:46.276 are probably because it's
00:41:46.276 --> 00:41:48.039 not finding service stack.
00:41:48.040 --> 00:41:49.999 Which happens sometimes.