Comparing memory usage between node.js object and go map[string]interface{}
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    3d ago 100%

    Interesting observation! The most simple explanation would be that it is memory claimed by the Go runtime during parsing of the incoming bson from Mongo. You can try calling runtime.GC() 3 times after ingest and see if it changes your memory. Go does not free memory to the OS immediately, but this should do it.

    2 other options, a bit more speculative:

    Go maps have been known to have a bit of overhead in particular for small maps. Even when calling make() with the correct capacity. That doesn't fit well with the memory profile you posted well, as I didn't see any map container memory in there...

    More probable might be that map keys are duplicated. So if you have 100 maps with the key "hello" you have 100 copies of the string "hello" in memory. Ideally all 100 maps qould share the same string instance. This often happens when parsing data from an incoming stream. You can either try to manually dedup the stringa, see if the mongo driver has the option, or use the new 'unique' package in Go 1.23

    2
  • github.com

    In the original proof of concept for ranging over functions, iter.Pull was implemented via goroutines and channels, which has a massive overhead. When I dug in to see what the released code did I was delighted to see that the go devs implemented actual coroutines to power it. Which is one of the only ways to get sensible performance from this. Will the coro package be exposed as public API in the future? Here's to hoping ♥️

    10
    0
    It's not like I just change things for the sake of change
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    3mo ago 57%

    There is a dangerously large population of devs and managers that look at themselves, unironically, as the gigachads pumping out ui "upgrades"

    Many of these fail to realize how disruptive it is. UI change is like API breakage for the brain.

    I have lost track of how many times I've tried to help an elderly family member with an app after some pointless, trivial, ui change. Only ending with them entirely giving up on using the app after the "upgrade" because the cognitive overhead of the change is beyond the skill that can fairly be expected for them 💔

    1
  • Putting Go's Context package into context
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    4mo ago 83%

    The context package is such a big mistake. But at this point we just have to live with it and accept our fate because it's used everywhere

    It adds boilerplate everywhere, is easily misused, can cause resource leaks, has highly ambiguous conotations for methods that take a ctx: Does the function do IO? Is it cancellable? What transactional semantics are there if you cancel the context during method execution.

    Almost all devs just blindly throw it around without thinking about these things

    And dont get me startet on all the ctx.Value() calls that traverse a linked list

    4
  • What is the best way to store data for exceptional cases among nested objects in arrays in JSON?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    5mo ago 100%

    Depending on your needs you can also break it into a columnar format with some standard compression on top. This allows you to search individual fields without looking at the rest.

    It also compress exceptionally well, and "rare" fields will be null in most records, so run length encoding will compress them to near zero

    See fx parquet

    1
  • What are some popular applications that use C?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    11mo ago 100%

    Postgres and MySQL/mariadb are all primarily written in C.

    Contrary to what other posters here claim, most programming languages are not written in C, but are self hosted. Ie. written using themselves. This usually involves a small bootstrapping component written in C or something similar, but that is a minor part of a whole

    2
  • github.com

    Go 1.22 will ship with "range over int" and experimental support for "range over func" 🥳

    14
    0
    Memoization in Go
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    1y ago 100%

    Pet peeve: don't use string keys. It invites key collision errors. Use the fact Go supports structs as keys in maps. Safer and more efficient.

    1
  • Intentionally corrupting LLM training data?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    1y ago 100%

    You should probably change page content entirely, server sizey, based on the user agent og request IP.

    Using CSS to change layout based on the request has long since been "fixed" by smart crawlers. Even hacks that use JS to show/hide content is mostly handled by crawlers.

    2
  • Go 1.21 Release Candidate
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    1y ago 0%

    I am super excited for this release. I think varargs min/max() built-ins are my favorite feature. Closely followed by clear() and improved type-param inference

    0
  • Does anyone here use a functional language in their day jobs?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearKA
    kamstrup
    1y ago 100%

    I have written a bunch of Clojure in previous positions. But it has undergone the same fate that almost all functional code bases I have knowledge of (in corporate product settings): Colleagues have hard times getting into the functional mindset, and it becomes hard to maintain. Over the years it gets replaced with some more pragmatic hybrid- og OO language.

    I have seen the same with projects written in Haskell, Erlang, and Elixir.

    It's all a really nice idea, but in practical reality it runs into issues with "social scaling"

    EDIT: Realizing this was not super helpful. If you want to look for positions where fp can be employed I think something academia related, or a startup where there is greater technical flexibility is something to look for

    1