orware

Should have checked before submitting. Seems like it was posted yesterday already: https://news.ycombinator.com/item?id=34526436

I'd be curious to hear about experiences with persistent pooling usage within PHP, since I don't feel that I've heard about it being advised as much myself over the years, but that could perhaps be due to old misperceptions about it.

I know there is the "Persistent Database Connections" section of the PHP manual and the mysqli extension within PHP supports connection pooling / persistent connections, but in my own experiences I've rarely seen them utilized, especially by the bigger open source projects out there such as WordPress, which has an 8 year old enhancement topic on the subject: https://core.trac.wordpress.org/ticket/31018. Putting your database behind a pooler, like ProxySQL let's say, is another option as the level of sophistication for a company/application increases, but most typical PHP setups I've used don't have that immediately available.

I've generally been under the impression that most projects/applications don't use the built-in pooling features for some of the reasons discussed in the link above, leading to those applications being more impacted by lengthier connection times due to a new connection being created at the beginning each request and then closed at the end of the request.

Now I'm inclined to experiment a bit with the built-in mysqli pooling feature though since it would seem a worthwhile feature for developers to experiment with more if it would lessen the connection time impact for each PHP request, particularly for databases that are further away and require secure connections.

Shaving off 100ms for a connection would be significant for most PHP users if they are currently having to open fresh connections on each request, especially if they were previously used to connection times of < 1ms when connecting to a local MySQL database.

I'm not an expert on the TLS 1.3 but the 0-RTT feature seemed like it wasn't implemented by a lot of clients so the new QUIC protocol used in HTTP/3 seems to be the workaround for that. The following recent comment and the first video linked actually had some great related info that I was recently reviewing on that topic: https://news.ycombinator.com/item?id=32572825

I don't know if the MySQL protocol itself though be able to utilize the TCP-based TLS 0-RTT functionality or not however so connecting via a regular client may still end up with a lot of the back and forth handshaking.

The newer serverless driver for JavaScript has some opportunities to take advantage of QUIC within HTTP/3 in the future as Matt mentioned over here recently: https://news.ycombinator.com/item?id=32513043

So that will be interesting to continue seeing how it evolves/improves over time.

You will generally run into different types of latencies in this case since you have the general connection latency when those are getting established plus the regular physical/network latency between where the database is located and your own servers.

For connections, since a TLS handshake is required, the impact of physical distances can have a greater impact on the connection time. The following article: https://sking7.github.io/articles/44961356.html actually provides a good 3.5x-4x figure which correlates with some connection tests I've completed.

In other words, if an initial TCP packet takes ~100ms to get from the database to your server, then establishing a TLS connection to the database will probably be around 400ms.

Once the connection is established, running queries over an open connection is generally going to be quicker, at least for simpler queries. More complex queries will still take whatever time they need to process on the database end before they can start sending results back so results will generally vary there.

But going back to that 100ms example...if the amount of data being returned from a very simple query is minimal than the response time here would be very close to that 100ms figure over an already open connection and likely would go up from there depending on the complexity of the query and amount of data needing to be returned.

Since the connection hostnames are publicly accessible and TLS is always required for connections you can easily test from your own provider's location. So long as the general physical location isn't too far away from a supported region, the latency overall shouldn't be unusable.

I may have mangled some terminology/analogies above but hopefully that helps provide a bit of a ballpark for you. If you have specific to/from regions in mind I might be able to try and collect some specific numbers for you!

That wikipedia article certainly brings up some need for pause/concern.

Recently, I heard about this relatively new company named Quaise: https://newatlas.com/energy/quaise-deep-geothermal-millimete...

And I thought that the technology seemed quite interesting, so I hope the development of it can pan out since it seemed like it would enable greater usage of geothermal energy around the world, which seems to be mostly untapped at this time, and also seemed like it might be able to address the seismic issue/concern, although I'm not an expert on the topic so I could be wrong there.

Geothermal is mainly a topic of interest for myself since there are a few local plants in my own region but I hadn't really realized that it is an underutilized power generation option until I read the article on Quaise recently above.

I'm not an expert on this topic, but one author in particular that I came across as a younger man was James Allen and his writings: https://en.wikipedia.org/wiki/James_Allen_(author)

And in particular there is a 3-volume series that was available for a few years that put together a lot of the work he had done during his lifetime, even though his most popular seemed to be "As a Man Thinketh".

The Wisdom of James Allen I, II, and III: https://www.amazon.com/Wisdom-James-Allen-Including-Prosperi... https://www.amazon.com/Wisdom-James-Allen-Difficulties-Trium... https://www.amazon.com/Wisdom-James-Allen-III-Heavenlylife/d...

It's been a number of years since I last read them, and unfortunately the publisher above went out of business so you can generally only find the titles above used, but I did enjoy the philosophy/thinking shared in the writing, even if their titles may indicate a somewhat religious slant, overall I'd say the writings focus more on leading a good life.

This is a good reminder I should read them again to refresh my memory on all that is discussed within the pages however, since it has been probably over 15+ years since I first read them thoroughly.

Quick question…did your account have a history and past positive reviews from any purchases you had made in the past?

I have a long-standing account and rarely buy on eBay nowadays (I’m trying to recall if I’ve ever sold anything…if I have it was maybe only one item but I don’t even recall if it sold or not).

Recently, I was looking into buying a used gaming PC via eBay to save a few bucks and I ended up completing a “Buy it now” purchase quickly without looking more into the seller (or their location). The location wasn’t a big deal (Paris, France) but that mainly meant the shipping would take longer. What ended up being more concerning was the 0 rating for the seller, which immediately make think “oh crap”. I reached out to the seller just to see if I could get a response with no quick reply, but I sent one short follow up the next day when I didn’t receive a response and shared my concern and waited another day before reaching out to eBay about the concern I had about the seller (especially because by this point the original listing was gone and then it even seemed like the seller’s account too). I used the live chat option and the person there was very helpful and got the process started and mentioned to reach back on Friday (about 3 days later), but later that same day my refund was issued and the case closed which I was grateful for.

But it did make me wonder of what might be an apparent difficulty for newer accounts to sell successfully on the platform? (Kind of like stories I’ve heard about liquor licenses being grandfathered in for certain locations in cities, whereas it may be more work for a new location to apply for one…maybe newer sellers can easily be flagged? The inability to dispute the situation when you are obviously willing/able to communicate with the eBay staff however is the sad part in your story since legitimate individuals should always have recourse to be heard in these large tech platforms).

Another note…based on what I know, most colleges are suffering from low enrollment too, and even though funding isn’t solely based on the the student count on census date anymore (which is usually about 1-2 weeks into a semester…and faculty is supposed to drop students that don’t come on the first day of class typically)…since now the formula is more complex with “success” related factors added (numbers of degrees / certificates awarded plus some other ones I don’t directly recall right now.

This leads to number of students still being a pretty big factor in the funding received for the year. From Administrators I would say if any are worth their salt, ignoring any sort of fraud would be a no-no so I’m hoping that’s not a common situation being observed. On the other hand, losing a substantial percentage of your current budget due to a loss of students can be pretty tragic for the staff working on the campus. Budget reserves can usually be dipped into for a period of time, but what most folks don’t know or realize is that compared to private businesses where the cost of employees may be only a fraction of what the business brings in profit, most CCCs are likely spending 80-90% of their budgets on salaries and benefits for their staff (in some cases the % may be more, in some cases it may be less). This makes it extremely difficult most of the time to weather a big loss in students because if budget reserves get expended, and student numbers don’t improve, that’ll mean some sort of layoff process…which also provides those employees with reinstatement rights too for a considerable period of time afterward).

XAP is the old system, but I don’t think it’s in use anymore (back then it was just “CCCApply”).

The current system, now called “OpenCCCApply”, was mostly developed by a team from Unicon (https://www.unicon.net/), along with folks within the CCCTechCenter team of my memory /understanding is correct (I’m not sure if it’s managed jointly though, or if only a CCCTechCenter team mostly manages it now though since I never heard those types of details).

A redesign/modernization effort however sounds…expensive.

It’s interesting to see this thread crop up here, as I’ve only recently left CCC system for a job in tech after being in IT within the system for about 14 years.

This particular issue over the last year or so has gotten worse, with more eyeballs on it, once actual money became involved (before it was still an issue but on a smaller scale mostly for the free edu emails we tend to issue, along with other freebies that can help enable, such as free credits for Azure or other services).

Even in the previous cases, I was annoyed/upset because in my mind the first line of defense the colleges have is preventing these fake users from being able to submit an application successfully in the first place, since the OpenCCCApply application (which I believe is used by all ~115 CCCs) was allowing the submissions in the first place…and since we mostly bring that application data into the individual colleges, not many triggered a “hold” on our end.

Yes, CCCTechCenter (which helps manage the team which maintains the OpenCCCApply system) have done a few things over this past year that are mentioned in the article already (but based on the article I can’t tell 100% if it is really indicating the issue is still rampant in the more recent semesters…one of the changes was adding usage of an IP reputation checker in for example, but there are likely ways around that too for these folks who actually don’t seem to actually use bots…maybe they use actual people instead based on what I’ve seen, such as the YouTube video shared in the article).

What I found really annoying by it all is that while the problem originated from the systems being provided to the colleges from the state level (OpenCCCApply mainly), the individual colleges are now on the hook to gather a bunch of mostly useless data, and go on silly adventures such as investigating IP address info within our other systems (like Canvas) to help find or report the fraudulent activity.

I think I saw FAFSA mentioned a few times but I don’t think there is a ton of fraud coming from the FAFSA application too directly…but in this past year many of the colleges have been putting COVID relief funds they’ve received (to help get students/staff back on campus) and using those to pay for fees or provide an extra amount for books, etc. which isn’t something that will continue forever (in fact, I think for this summer this will already have ended, or it will be the last semester where it will be offered).

In most cases, once the incentive is taken away, or the bar to get it is made higher, these folks creating the fraudulent accounts will generally move on (or target schools that don’t implement some of the 2nd layer fixes at the college level…unfortunately while the CCCTechCenter tries its best, it doesn’t typically fully acknowledge its role in creating some of these situations, and I almost lol’ed when I saw towards the end of the article I saw they are looking to get more funding to “modernize” it, yet again, considering a lot of effort / time / money already has gone into creating the current OpenCCCApply system not that long ago from the previous system, which was pretty bad in comparison).

Overall though this particular situation is at the same both more complex and simpler than folks may think, once you have some more details (more complex because there is a lot about what’s going on in the CCCs the HN community isn’t aware of, along with super strict regulations that have to be followed within the individual Financial Aid departments at each school, otherwise they win not be able to provide federal aid monies to students if they weren’t doing so, making that avenue for fraud a lot more less likely than the scenario I shared above on how the COVID relief monies have been being used instead to provide an incentive to get students back in the classroom…along with the solution being simpler since we already have a central application process that should be the system that keeps these applications from ever reaching the individual colleges, but it fails in that regard…that along with removing the financial incentive currently present, should reduce the fraud levels considerably…although there are likely a few more complexities even I am unaware of…I would just appreciate it if the search for fraud wouldn’t get pushed onto the individual colleges in these situations where a system wide protection should have prevented the situations in the first place, mainly because it causes a ton on unneeded busy work at the colleges keeping IT System Analysts and other technical folks from focusing on other, probably more important, internal projects).

Excuse any typos…I wrote this small novel on my phone.

It's still a bit early for the company but if you like RDS, PlanetScale might be worth a look too: https://planetscale.com/

Check out my comment above here in this same thread which basically confirms exactly what you just shared here...this is the same tactic I was observing being used with fake student accounts sharing with outside external Gmail addresses which then appeared to take over the Shared Drives being created in GSuite, allowing for TBs and TBs to be added in to the drives by outside users. It seems like a big problem to me...but at the moment I don't think Google is providing good enough tools for Admins to see/eliminate these situations very quickly/easily.

Just coming out of a higher education position I held for about 10 years...we were really only using GSuite for Education on the periphery of everything else we were doing (which was all Microsoft-based). So we were really only setting up the needed GSuite integrations to give some access for students/faculty/staff to use the GSuite apps if they wished, so the "unlimited" storage side of things was an extra bonus.

While our actual students/staff have not abused the unlimited-ness at all, there's been a big trend in recent years for fake spam/bot type accounts to submit online applications (particularly for the inexpensive community college level here in California) which would allow for those fake people to get issued accounts within the college systems (typically giving them a student email account).

In essence though, one bad account could in turn create multiple shared drives it seems, and then from there grant access to other external Gmail accounts and swap the ownership to them, while still leaving the unlimited storage capabilities in place (at least that's been my perception of what I had seen in our instance). Before I left recently, I had seen pages upon pages of shared drives in our GSuite Admin area of Shared Drives that were terabytes large (one as high as 100 TB or some ridiculously high number like that)...with very little mitigation options being provided by Google to help avoid that from occurring in the first place (aside from taking the extreme measure of locking things down after the fact...but cleaning up that existing mess seemed to not have a solution...supposedly there were going to be some new tools released early this year to help out, but not sure if those are available yet).

Suffice to say...some bad apples are making it hard for all of the good people to take advantage of what was otherwise a wonderful offering for educational institutions (having administered many systems over the years...it's always nicer to not have to worry about license limits and other baloney like that since it simplifies things for what is usually an already overloaded IT staff).

I could clearly see all of the bad accounts abusing our instance...if only there was an easy way to filter all accounts > 2 TB and then delete them I could have reduced our storage usage down considerably for Google.

That multiplied by hundreds of institutions where this may have occurred during the pandemic and I can see how storage usage could have went up dramatically on Google's end...but I think that Admins in those institutions would be more than happy to clean things up if they had the tools/ability to do so.

I'm not sure I understand the criticism in the top comment in this thread about Caddy's performance.

I don't think I ever really decided to use Caddy solely based on performance itself (although already fairly familiar with Go's built-in libraries for creating HTTP servers gave me confidence that it should be fairly quickly...I wasn't overly concerned about how it directly compared to Nginx/Apache/IIS/etc.).

For me, the biggest selling point was the Caddyfile format, which felt a lot more human readable than the .htaccess or Nginx configuration options I had seen over the years (I know with Caddy v2 there's the possibly now more preferred JSON option, but I'm still a sucker for the original Caddyfile format myself :-).

There's still a lot I don't know about for all of the web server options out there in general so I don't claim to be an expert in any of them, but Caddy was the first one I felt comfortable using new situations to simplify things in my environment.

For example, Caddy's reverse proxy functionality (and the way it is setup within the Caddyfile) is what finally made that particularly capability "click" for myself over 5 years ago when I first came across Caddy (even though I had seen info how Nginx + Apache were used together in the years prior, with Nginx providing reverse proxy access to Apache in those hybrid setups, it wasn't something that had "clicked" for me being aware of those situations). Since my environment is fairly simple (no real super high demand situations requiring lots of load balancing) we mainly use Caddy to provide automatic SSL capabilities and act as the publicly accessible endpoint in our environment, and use the reverse proxy functionality to tie Caddy into the various other internal servers that need that SSL termination.

I've also been able to take advantage of the built-in static file server for an internal website need and it's actually something I really think is a nice/neat feature to have available (especially due to its ability for simple server side includes, allowing for relatively easy simple templating).

Separate from that, I've also experienced first hand the personal responses that Matt Holt has provided to myself and to many others in the Caddy Community, and I'm definitely appreciative of the work he's put into Caddy (along with others of course) and I hope some of negativity in this thread is able to be easily shrugged off by Matt, because I could imagine it can be exhausting to deal with when he genuinely does care and puts a lot of thought/effort into the responses he provides.

Just an FYI, it looks like your 2-year GoDaddy Wildcard SSL Certificate expires today (Chrome sent me to the "Privacy Error" page when trying to visit your site a moment ago so I took a quick look at the certificate).

I would love to just focus on dozens of problems / interesting areas to myself, but those of us without healthy bank accounts don't really have that luxury.

I really wish there was a sponsorship model of some sort where folks that just love programming for the sake of it could be sponsored by those wealthy enough not to worry about money anymore.

While there are still a lot of interesting problem areas within a typical workplace, sometimes the interests one has within programming don't necessarily align with the workplace's needs, making it so that to work on the interesting stuff in your own time, which gets harder as "life" happens and bills become plentiful.

I too though read about those famous folks in science and noticed how they had either financial sponsors, or were themselves already wealthy, and that allowed them to pursue their own learning paths in earnest, where I would say that isn't an option for the vast majority of us today (it also circles back to general opportunity...there are so many smart and intelligent people out there that may never get the chance to reach their full potential due to the lack of opportunity and instead have to focus on more basic needs, etc.).

Although my unique circumstances are different, I'd like to share that you are not alone in feeling trapped in one's job and city since I too have felt the exact same way in recent years.

I unfortunately can't offer very many solutions since I am in basically the same "trapped" situation currently (but towards the end here I can hopefully share a few things that keep me going).

For myself, the position I currently have in my hometown is the best I can hope for in this area, but the work/team environment have steadily become worse, particularly the last few years...being able to work from home last year helped out tremendously since it was something I had been hoping to have for some time, but the main kicker was simply being able to be around my kids more and have those smaller interactions with them on a more frequent basis and not deal with the office drama constantly.

Age-wise, we're the exact same, but education-wise you're further along than I'll likely ever get (I have some work towards a Masters in CS, but I'm poor at math, so with a completed PhD in CS in hand for yourself, I feel that opens up so many more doors than I could hope to receive). Housing-wise, I fortunately live in a somewhat rural area so housing prices aren't nearly as high as they are in other parts of California so we were able to take the plunge to purchase a home about two years ago (at times I feel that renting was definitely easier, but we were outgrowing our old place so I echo the sentiment of having one's own place to make "home" as being a positive).

I think the big thing I'm reading in your post above though is potentially that proximity to family and a city where you actually do feel like you're home and I can also echo that sentiment too. I spent a year in the Bay Area for school, but always felt that it was different than my home area, and later on during a short stint at a tech company in the Bay as well, those sorts of feelings returned too (it's difficult for me to say though whether I might have eventually felt "at home" if I had given myself more time but I'm not sure if that would have occurred or not).

In the last month my workplace has had myself return to the office and it created an immediate sense of sadness since I would no longer have the comfortable environment I had built up at home, nor would I get to see my children as often I had grown accustomed to. That I feel is the great drawback of so many companies pushing to bring their staff back onsite rather than adopting proper remote work options and allowing for it moving forward.

On top of that, I took a short (less than a week) vacation with the family, and had similar feelings as yours in my first day back in the office afterward since I just resented the thought of coming back into the office again so much after enjoying a few days away.

Also like you however, I've even shied away from wanting to apply for Software Engineering roles at other companies because I have the same concern regarding whether I would prefer the new work more than my existing work...or would it in fact be worse? That's the big concern (aside from the larger questions of whether remote work would be allowed and if I would be forced to make larger changes such as completely moving my family away to another city, etc. which also doesn't seem as appealing to have to put them/myself through either if it can be avoided).

So I wonder, what's my alternative to this line of work I've been in over the past decade+? And it's a really sobering thought since there would basically be nothing (that I can think of) that would come close in terms of salary so I feel forced to continue on in it (even though I do enjoy software development overall, I've also worked mostly independently since I'm basically a team of one, so I'm never sure if working in a larger software oriented organization might take away some of the enjoyment I've had working independently on software projects due to the extra rigor that might be needed on larger teams...overall though I'd still hope that the extra comradery would be a net plus on that scale).

Since you mentioned that your partner earns several multiples of your current salary I'm not sure what kind of accelerated timescale that might provide you as a couple to a potentially early retirement elsewhere (potentially closer to your hometown), but that might be consideration (although it doesn't help out with the here and now feelings which can't be worked around if you're still needing to go to work and experience those feelings daily).

For myself, it is difficult too...in years past what would sometimes help is me making attempts to get hired at tech companies (providing some small hope for that short period of time before being eventually rejected...which is also a not so great feeling so that's one reason why I've shied away from doing much of that lately), but at the moment I'm actually exploring Y Combinator's Cofounder matching area to see if that might provide me one path away from my current role I feel so stuck in and maybe take me into a new role (e.g. CTO) where I can provide more impact in a smaller company and get to stretch more of my brain and what I feel I can bring to an organization. I'm not sure if this experiment will pan out or not, but I'm hopeful for now :-).

Feel free to reach out via my email in my profile if you feel like sharing any more of the "struggle" we both seem to have (it's good to know I'm not alone in feeling that way either :-)!

I had been trying to get into Tailwind CSS in recent projects but noticed some of the Bootstrap functionality I tend to like was missing out of the box so with Bootstrap 5 coming out last week I switched gears to start using that for a new project.

The project itself is an update to an older one that was originally built with Bootstrap 2.

For the old one, when I go to print the somewhat detailed tables I'm using show up fine, but in the new Bootstrap 5 version, one of the tables sticks outside of it's container when going to print and for the life of me I've been struggling to figure out a solution that works consistently for the print side of things (which I feel tends to be overlooked quite a bit in regards to CSS). What's even odder, is that when I implement a change that helps with the second table (making it appear contained within the page dimensions) the first table above starts exhibiting the issue of it being outside of the container. Also, in this case, if I were to leave the default styling of the "table-responsive" div containing the table, you wouldn't be able to see the rest of the columns at all when printing (since the scrollbar would show up) so everything I'm describing above is with an override to have "overflow-x: visible" when printing. Some other suggestions online included adding some overrides that use "page-break-inside: avoid;" for tables/tr/td/div elements and "display:block !important;" for the body and div elements but I'm not completely happy with those results, but yup that's been a bit frustrating trying to sort out since I'm not even sure why it's behaving the way it is at the moment.

I haven't kept up fully with all of the Sass/SCSS advancements in recent years but have a semi-decent CSS foundation to build off of (from my understanding those advancements primarily help with some additional programmatic elements that then get pre-processed back into CSS? Along with some syntactic sugar to simplify some of the repetitiveness that CSS can require).

The other big annoyance I was dealing with was related to what appears to be a long-standing WebKit issue related to scrollbars on mobile devices? (for the theme I'm using, I'd scroll down on an iOS device and wouldn't be able to see the very last item in the mobile menu). I found a reference somewhere (probably Stack Overflow of course) that mentioned using "@supports (-webkit-touch-callout: none)" to try and target the mobile browsers specifically and I reduced the height by an extra 65px or so which seemed to be enough to help with that particular issue (the theme utilizes "perfect scrollbar" it seems, but not entirely sure about all of the styles involved). (Before settling on this solution I had already tried out the solution described over here: https://allthingssmitty.com/2020/05/11/css-fix-for-100vh-in-...).

One last item I was dealing with this week seems to have to do with SVG rendering on the desktop vs. on mobile (for a logo). We sort of have a designer on staff but he deals more with print than with web stuff so I asked for a variation of our logo with the name of the application integrated into it and provided back to me as an SVG (since the PNG I had created myself looked slightly fuzzy when it was downscaled to fit the space I was using it in on the page, so I felt an SVG version would look crisper). After swapping things to the SVG version provided to me, it looks pretty good on mobile, but on the desktop (both in Chrome and Firefox), things just don't look as good. Some resources online mention it could be related to the lower DPI of a desktop monitor vs. the retina type DPI of a mobile device, but I don't know...I also saw some mention of a "pixel hinting" approach that might be able to be utilized by the designer in Adobe Illustrator perhaps that might help as well so that's something I've asked him to give a try for me so we can see the results, but I was kind of disappointed about the way the SVG looked as well (and now that'll require a bit of extra time to research/learn/experiment about, rather than focusing more on the application I'm trying to actually build...but at the same time it's semi worth it at the moment since I'm hoping the foundation I'm building with this new web app will be utilized in a couple of other updates that I'd like to make over the coming year, but we'll see how it goes ;-).

I'll postscript this message that I'm not a frontend dev, but since I'm basically a team of one with an eye for certain details/issues, I try and get these sorts of problems sorted out at the beginning of a project so they don't cause issues, but also as a team of one, I really wish that in addition to the backend frameworks that are available (Laravel in my case is the one I'm currently ramping up on, as it's the more or less leader on the PHP side), it would be nice to have a nice frontend option with "batteries included" for a developer to really utilize and not have to worry too much about (Bootstrap has probably been the closest in that regard...I do like a lot of what Tailwind CSS is doing though, and of course it's heavily promoted in the Laravel community, but I just found that it would require too much work for me and too much time...and my workplace isn't exactly software development focused so I have to pick my battles in that regard vs. spending a lot of time getting something basic going vs. getting something shipped for users to start using).

Hopefully one day we'll be completely away from needing to do any sort of CSS hacks (and things have come an extremely long way in the past few years), but at the moment I wouldn't say things are completely hack free yet (at least for us non-100% frontend folks...you all might have some extra magic you can cook up on the fly to solve any problems encountered maybe using what's already available :-).

As an aside (or perhaps related note), over this past weekend I was doing my first real-world usage of SQLite, since I typically rely on conventional databases for my day-to-day work.

I've read the article shared here in this thread in the past and read about the usefulness of SQLite generally, particularly for read-heavy situations, so I thought it would be a nice experiment to pull in the "Have I Been Pwned Password" List and import it into the SQLite and see how fast it could be queried.

At first I didn't find too many resources out there of the same thing having been done, but after searching some more throughout the weekend I did find out some nice nuggets of information that helped (in particular, I learned that adding the "WITHOUT ROWID" option at the end of a SQLite CREATE TABLE statement will help to reduce the overall database size considerably).

The desire to put things into SQLite for the "Have I Been Pwned Password" actually stemmed from a misinterpretation of the API page since it seemed at first that the API was not free (although I had thought it was), since one of the first things I read was the need for a $3.50/mo charge. Later on, I then figured out that the password checking API is still free, it's just the other pieces of the API that require the API Key / monthly charges (doh!).

If you run a query against Troy's API, it's pretty damn fast all things considered, so I'll likely revert to using his API directly, but I was thinking of uploading the SQLite versions to some cloud hosting somewhere and potentially share it with others if anybody is interested.

On to some quick stats collected over the weekend (using an import process powered by some PowerShell):

  50 million took ~6 hours 22 minutes to complete (with ROWID):
  Start Time: Sunday, February 14, 2021 9:12:29 AM
  End Time: Sunday, February 14, 2021 3:34:46 PM

  50 million took ~5 hours 46 minutes to complete (without ROWID):
  Start Time: Sunday, February 14, 2021 7:51:38 PM
  End Time: Monday, February 15, 2021 2:05:30 AM

  5.28 GB (50 million rows, with ROWID)
  2.61 GB (50 million rows, without ROWID)

Example performance of Select-String (line-by-line search):

  TotalMilliseconds : 216245.7246

  TotalMilliseconds : 20.8543

Example performance of this approach was more in line with the SQLite database:

  TotalMilliseconds : 40.3749

  TotalMilliseconds : 4.9491

Tagging along with the top comment in this thread however, I do see what they mean about insert performance since I did try a few experiments there as well, but went ahead and kept things with the 10k batch inserts at a time (I think with individual inserts I was calculating things to potentially take 216 days or something ridiculuous, and switching to batch inserts made it go down to about 3 days, but based on some comments I read elsewhere over the weekend, even that seemed like it might be kind of high...at the moment I'm trying a full import outside of my PowerShell script using DBeaver's functionality for doing so, so I'll see if it completes faster...seems like it might since based on the current database's size it seems like it's more than 50% complete and it's probably only been about 12 hours).

I arguably have a pretty good job currently in the public sector which has a fair amount of autonomy and prestige that have been earned within my current organization over ~13 years.

Last spring after COVID hit I was reached out to by a Facebook recruiter with the conversation morphing after a few months (and two other individuals on their end) until having a really exciting conversation with a recruiter in early August that was just refreshing since it actually seemed like Facebook's hiring process wasn't that bad and that they might actually value me and my existing experience.

Part of my challenge is that my current role is not completely coding focused so I really don't have that "true" software engineering background and find it hard to figure out where I'd fit in a typical tech company's role structure. Instead, I have experience coding my own projects for my organization and taking them from concept to production in weeks and maintenance over timespans of years, along with management and tech leadership experience within my own organization.

Since coding isn't a daily focus all the time, my programming skills tend to need to be dusted off each time I take on a new project, and I know I'm not the best algorithmic expert, etc. but I'm reasonably intelligent (in my own humble opinion) and able to learn from others when I have that opportunity available (learning about extreme scaling techniques and certain advanced topics that might only be encountered within a tech-focused organization is unfortunately something I can't easily do within my current organization, and team-wise I'm the primary person interested in these sorts of topics so don't have others to really discuss the ideas with, which leads to that interest in potentially moving to a tech-focused company). I feel like my existing experience and ability to work with others and learn quickly would have some value (especially when you consider that new graduates don't necessarily have a lot of these experiences under their belt, but they are still considered for hiring...their algorithm knowledge might just be a bit fresher/recent).

Switching back to the interview...I spent a bit of time studying for the initial coding interview and felt like I had done decently well, but still received the "thanks but no thanks" email a few weeks later.

After that rejection (along with others over the last several years from companies I saw on the monthly "Who's Hiring" threads that intrigued me) I'm kind of done with trying to apply for tech jobs at the moment.

I'd love to earn more and work remotely from my current location (now that the pandemic has made this a little more possible) and be able to contribute my working days to contributing to the success of a tech-focused company, but it's not easy to be accepted by a tech company when you don't have the right background or exact skills upfront, so at least for the time being I need to be happy with my current work situation (even if it isn't always the most glamorous, tech-wise, and doesn't pay the same as a large tech company might).

This is just one personal anecdote of course, but I do wish there was a different hiring process for folks wanting to break into a tech company with existing experience that would be valuable, albeit non-traditional to typical hires.

Not that many people may be using Oracle and PHP together, but last summer I decided to take a closer look at some performance aspects since I'm wanting to move certain PHP things to the cloud, while our Oracle database remains local, which would introduce some extra latency.

In some basic testing on my local machine at work just connecting to our local Oracle database introduced a good 60 ms - 100 ms for a request (before running any sort of queries) and due to PHP's nature this recurs on every new request.

I had searched for options for a long time in past months/years that would allow PHP to have some sort of connection pool the same way languages like Java/Go might allow, but had come up short each time.

I don't even recall how I came across it, but last July I finally learned about Oracle's Database Resident Connection Pooling feature (and then reminded my coworker about wanting to try it out in November) and was able to run some tests and it does a good job of eliminating most of that unnecessary connection waiting time for PHP applications so I was pretty happy with that testing and it didn't seem to increase the load much at all on the database server side.

Even though it might not be as ideal as a truly separate Connection Pool being managed at the application server level, or outside of the database, a Connection Pool available for use at the database level still has benefits so it would be kind of neat to maybe see more of this sort of approach be available with other vendors (if it isn't already).

EDIT: Forgot to add that as a result of the DRCP testing, even though moving the PHP side to the cloud introduces extra latency, with the time savings of the DRCP change, it helps reduce latency enough that overall it results in a time savings over the normal/conventional connection approach when the PHP code is local so for me that was a win (hopefully I'll get to try using it in production in the near future, but waiting on my colleague to sort out a few things with enabling it in production at the moment).

Not sure if Maddy covers this use case specifically (I didn't see it covered in the documentation, but I figure it actually is possible).

I'm hoping to use a Golang-based application as a simple SMTP Relay, which also allows for ranges of IP addresses to be whitelisted to connect openly to it from within our campus network.

Basically, the scenario is as follows: a number of internal services are older and may not support the authenticated route so we can use Sendgrid directly and send emails securely, so instead we currently have them routed to an internal Windows server which is running an older piece of IIS functionality (it's an SMTP Server in Windows that can be turned on). That piece is configured to be open (so it can receive messages on Port 25) but is also limited to a set of IP address ranges that internal servers are using (so that it's not generally usable by just anyone inside our network) and is configured to send those received messages out using Sendgrid.

I found the following Goland project just a few months back that seemed promising: https://github.com/wiggin77/mailrelay

And I'll probably need to follow up with the developer again (or try and make the contribution back to the project on my own), but the main issue it had was that the IP range whitelisting wasn't available (and even whitelisting by specific IP addresses seemed problematic, so in my testing I had to keep it completely open for it to work as expected).

If Maddy can fulfill the need instead though I can give it a shot ;-).

I signed up for it too last weekend after coming across it after doing some research (I had been making a bunch of video recordings a few days prior, and once the videos were added into Camtasia and the audio played back I noticed a lot of background hum coming from my HVAC return outside of the room I'm in).

Was impressed with the Krisp.ai tech as well and probably works similarly to this tool and the other Nvidia solution that I can't try out since I don't have an RTX card (main difference might be the overall training set that Krisp has already run their algorithm through?).

I haven't had any Zoom meetings since purchasing Krisp, but I had been using the built-in mic from my LG Tone headset for those meetings.

Since making those video recordings I've been using my blue Yeti mic (and a pair of headphones connected to the mic for listening) as my primary and I've continued running a bunch of small tests to try and see if I can be happy with using Krisp enabled all the time.

Currently, I don't feel comfortable with leaving it on all of the time though for recordings, particularly with something like the blue Yeti mic which is able to capture pretty rich audio. In my testing, Krisp did a great job of eliminating the background HVAC humming noise, but replaced that issue with two others: some minor (but distracting) hiss/noise between words as I'm playing back the recorded audio, and also currently is limited to 16000mhz frequency (not sure if mhz is correct or not in this case...this is what support shared with me when I asked about audio quality degradation). The support person did respond though and say that the team is working on the increasing the frequencies they are able to work with though so I guess there might be some improvements in the near future on it?

After seeing the latency figures on the NoiseTorch page it makes me wonder if the Krisp latency is similar or not (so far I haven't noticed any latency issues with Krisp).

As far as remaining thoughts...I kind of wish there was a bit more configuration options available for Krisp, but the simplicity of it is also a benefit (for others that might not be as technical and just want a simple solution that does appear to work overall). I haven't gotten it to work for playback needs (it has the toggle for it, but nothing seems to happen when I try and toggle that on). Also, still not sure what the overall differences/improvements with Krisp Rooms enabled (I am recording in a room, but after reading their description/blog announcement page it kind of seems like it's more for conference rooms where multiple people are speaking and extra echo cancellation might be useful? ref: https://krisp.ai/blog/krisp-rooms-launch/)

Since I'm already out with a year subscription with them I'll continue to try and figure out how to use it effectively, but not as excited about it at the moment compared to how I was last weekend initially (impressive overall though...hopefully it continues to improve :-).

Does your solution allow for Administrator Escalation assistance on Windows?

Over the past few days we've been transitioning our higher ed staff to working remotely and are able to use Microsoft Teams or Zoom already to see another staff member's screen and walk them through basic things, but if the user needs to install a program, which would normally require admin rights on the machine, it appears that neither of those two solutions allow for that type of assistance from our IT staff.

Does Screen provide a solution for that situation? If so, we can give it a shot with our IT staff ;-).

I'm not an expert, but I had heard lots of good things about Fast.ai's online course/content: https://course.fast.ai/

I've started/stopped a few courses with Georgia Tech's OMSCS program as well which might have been useful, but I still feel like I'm missing some of the mathematical foundation to allow me to make more sense of those courses so Fast.ai's approach seems like it could be a better fit for someone like myself that's more interested in the practical aspects of using it (I just haven't made the effort to go through their content myself).

I think these programs are great, and the formerly incarcerated do need more of these types of opportunities otherwise it will only continue the cycle of recidivism.

On the other hand, I also wish some of these types of opportunities could also be offered to some of us that aren't part of the big tech companies currently but have talent and drive too, but might have a poor or limiting work environment that they can't easily withdraw from.

I'm sure many people with coding experience outside of the tech industry that are interested in breaking into the "tech companies" in the Bay Area would appreciate getting that opportunity to demonstrate their skills, but if one's background is not appreciated and you can't even get to that point to show a company what you're made of then that also seems like wasted opportunity for companies that are missing out on good potential talent.

This past year I've been very much interested in joining a fully remote/distributed company but I would very much appreciate getting that opportunity to work with a team and start showing what I'm capable of, than harping on the fact that I haven't been resolutely focused on the latest language, framework, or specific tech these past few years (it doesn't mean I'm not aware of most of them, or have dabbled with each of them to a small degree...just not enough where I can tell you with confidence I'm an expert...plus any new role I would jump into would likely require it's own specialty that I could acquire within a short time of starting.

Got to give us "normies" an opportunity too ;-).

The full article has a cost associated with it, but it looks like the writer has also given this past speech with (hopefully) similar content:

https://www.youtube.com/watch?v=ellIRUe2mpU

HN Hug of Death?

I've looked at ORMS mostly on the periphery for years and years now, but haven't really been able to use them in our environment and this might be similar to the situation for others as well.

One thing I noticed early on, is that even using a query builder complicated things a bit as well since it meant that the original SQL string was then broken up into multiple method calls to construct the SQL string. Since the full query wasn't built until the very end, this meant that a helper line needed to be added into the code to get that full SQL query if any issues were found down the road and then taken over into our program of choice (in this case, SQL Developer, since we're dealing with Oracle queries in most cases) and running our additional tests over there.

Our campus ERP is pretty complicated table-wise, so our queries (developed either by myself or by our systems analysts) tend to be fairly complex, requiring multiple joins and other complex logic that I feel would have a very difficult time being translated into an ORM format.

A query builder is still somewhat usable, but for the most part I just stick to mostly straight up SQL queries and make use of prepared statements to help avoid SQL injection and keep life simple so it's easier to move back and forth between the application side and testing the query on the database side :-).

On a side note, I'm not sure if it's mentioned here in the discussion (and it might be less of an issue now than in years past) but I have noted some ORM usage doesn't always choose the most efficient mechanism for things (e.g. pulling results using small, but relatively expensive multiple SQL queries rather than being contextually savvy enough to know that a set-based query would be better suited to retrieve the entire set of results at once).

Overall though, working with SQL and coming up with solutions for things is probably one of the funner aspects of my current position and while using an ORM has sounded like fun in the past, it just hasn't seemed like it's the best fit for our particular workflow/environment.

It can definitely be difficult working full-time with a family. I hit the halfway point in the program in Fall 2017 (after starting in Summer 2014) and I haven't been able to make any progress since then.

I know in order to do well in the classes I sign up for that I need to spend the needed time to complete the work, but it's been super difficult for me and balancing time with my wife and kids and my work.

I really wish they could launch more self-paced options for the courses, rather than keeping strictly to their on campus semester schedule. To stay active in the program I have to keep on signing up for classes, but then end up dropping them over the past few semesters.

On the one hand I want the masters degree, but on the other hand it's difficult to force myself to stay motivated and interested in the classes since not all of them are fun/enjoyable (or allow for the time to learn in a more self-paced manner).

I'm also not a big fan of online testing / proctoring...I would rather just complete projects ideally.

There've been other students that have been successful in the program even with full-time work, signficant others, and children in the mix, but those first few years in the program were definitely a sacrifice personally and it's been difficult to continue with that same level of sacrifice since the end of 2017 for me personally (hence my wish for a more self-paced option where I could accelerate/decelerate my own progress since the tie to the regular semesters really slows down my ability to make progress as someone that has difficult fully dedicating the time regularly now).