acutesoftware

and maintain

This right here is the key difference! Yes anyone can vibe code a replacement for many apps - but will it still run 2 years later (assuming they get it 'running' in an prod environment at all

This highlights that all RAG systems should be using metadata embedded into each of the vectorstores. Any result from the LLM needs to have a link to a document / chunk - which is turn links to a 'source file' which (should) have the file system owners id or another method of linking to a person.

If the 'source information' cannot be linked to a person in the organisation, then it doesnt really belong in the RAG document store as authorative information.

Nice to see old tech revitalised - I had fun with the Australian version of a Z80 single board computer - https://en.wikipedia.org/wiki/TEC-1

I am using LangChain with a SQLite database - it works pretty well on a 16G GPU, but I started running it on a crappy NUC, which also worked with lesser results.

The real lightbulb moment is when you realise the ONLY thing a RAG passes to the LLM is a short string of search results with small chunks of text. This changes it from 'magic' to 'ahh, ok - I need better search results'. With small models you cannot pass a lot of search results ( TOP_K=5 is probably the limit ), otherwise the small models 'forget context'.

It is fun trying to get decent results - and it is a rabbithole, next step I am going into is pre-summarising files and folders.

I open sourced the code I was using - https://github.com/acutesoftware/lifepim-ai-core

I am working on a local RAG LLM designed for lower end PC's - ability for people to try out searching their own documents, seeing it was such a learning curve to get to this stage - hoping others can learn from my mistakes.

https://github.com/acutesoftware/lifepim-ai-core

Only been public a few days, so please let me know if there are glaring issues.

Switching models when running locally is fairly easy - as long as you have them downloaded you can switch them in and out with a just a config setting - cant quite remember, but you may need to rebuild the vectorstore when switching though.

LangChain has the embeddings for major providers:

  def build_vectorstore(docs):
    """
    Create vectorstore from documents using configured embedding model.
    """
    # Choose embedding model
    if cfg.EMBED_MODEL.lower() == "openai":
        embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    elif cfg.EMBED_MODEL.lower() == "huggingface":
        from langchain_community.embeddings import HuggingFaceEmbeddings
        embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
    elif cfg.EMBED_MODEL.lower() == "nomic-embed-text":
        from langchain_ollama import OllamaEmbeddings
        embeddings = OllamaEmbeddings(model=cfg.EMBED_MODEL)

I think it was the modal dialog box that forced you to stop what you were doing and click 'piss off clippy', rather than being able to ignore it.

Some people love programming, for the sake of programming itself.

And this is what is causing the friction against LLM's (which are quite useful for getting up to speed with a new concept / language ), the programming itself is the fun bit - I still want to do that bit!

Very well done - it took me several attempts to get the hang of Blender and I still only know / used half those shortcuts. Thanks for the easy to use tips!

I think Google has lost all faith in terms of keeping projects around - especially when they involve data locked into a mildly complex system without a complete migration path out.

I would be wary investing time in learning / using any new products from them.

Cyc seemed to be the best application for proper AI in my opinion - all the ML and LLM tricks are statistically really good, but you need to parse it through Cyc to check for common sense.

I am really pleased they continue to work on this - it is a lot of work, but it needs to be done and checked manually, once done the base stuff shouldn't change much and it will be a great common sense check for generated content.

Our ETL process is heavily monitored so we never miss a days data, but we got a surprising error "cant build aggregates - missing data, aborting MV refresh, data will be a day old". It was the year to date (YTD) calculation - no data for 29/2/2023 to compare to today.

"You can't just take the CSS stylesheets and references and store them."

Yes you can - just save them locally?

Well, you can - you can have a policy that forces payments from a select group. Some users will complain when "No, you cant pay with 2 chickens every 3rd full moon", but that is too bad.

Oracle requires you to login to get access to most support documents now.

https://www.lifepim.com/blog

Very low frequency updates mainly about information management and data.

Most popular article was "What software will you trust when you get senile?" - https://www.lifepim.com/blog/5856_What_software_will_you_tru...

This is really impressive, even the trees look pretty good which appears to be tricky to do with procgen stuff.

A nice next step or addition would be to take the results and remesh them to lower poly models so it can be used in a game engine to walk around in.

In terms of Unreal blueprints they may be low syntax but they are not low code - you still need to know software development techniques or you will end up in a pile of garbage very quickly (which you can also do with any other language).

1. Lots of additional data breaches will make more users wary of using cloud apps for personal data. This will cause more power users to move to desktop / local first apps.

2. AI will improve and showcase several new cool 'tricks' but commercially it will continue to focus on guiding users to watch more ads.

3. An increase in the number of streaming media services will make the average user realize they need to pay quite a lot money to 'watch stuff' - either a new unified package will appear for a reasonable price, or piracy will increase.

This is very cool, though as others have said it is fairly resource intensive.

I'd love to see a schematic exported or even shown as you press simulate (you clearly have it the components and links in memory somewhere to do the simulation). This would be great for debugging.

Not that recent, but another 2 good games are: Evolve ( https://pmotschmann.github.io/Evolve/ ) start as cells, evolve to humans (or anything), craft stuff, conquer space

Swarm Simulator ( https://www.swarmsim.com/ ) breed bugs, eat meat, consume universe

Oh that's just great - I was planning to muck about with some odd coding tonight, but I guess I'll be playing this (awesome) game again.

I think this is useful in that you don't need to use any specific render package.

It is nice to have a baseline of materials in an open accessible form.

The sounds generated by this are very impressive - it is an actual simulation of the entire combustion process.

Best parts are the engine sound under load, slight delay when opening throttle and changing down a gear to force engine braking.

Blender is the pinnacle of what Open Source software applications can achieve.

It started off with great features, but really difficult to use (for non artists), yet has gotten much better over time.

The number of features grows faster than I can think of uses for them, but it is nice to know they are there.

No notifications at all - phone is on silent at all times.

Though I do have a good habit of checking messages and emails every hour, so it isn't that bad for others. (Desktop is different, teams / slack is on all the time with popups).

### Disk and Hardware

Hardware Summary

    sudo lshw -short
    

    df -h .; du -sh -- * | sort -hr
    

    lsblk

    uname -a
    > Linux TREEBEARD 4.4.0-98-generic #121-Ubuntu SMP Tue Oct 10 14:24:03 UTC 2017 
      x86_64 x86_64 x86_64 GNU/Linux
    
    lsb_release -a
     > Distributor ID: Ubuntu
     > Description:     Ubuntu 16.04.4 LTS
     > Release:         16.04
     > Codename:     xenial
 
 What flavour of Ubuntu are you running (see https://itsfoss.com/which-ubuntu-install/ )
 
     cat /var/log/installer/media-info
     > Unity
   

     uptime
     >  23:09:26 up 61 days,  8:28,  1 user,  load average: 0.82, 0.48, 0.34

    find . -type f | wc -l

Get a tree view of subfolders

    ls -R | grep ":$" | sed -e 's/:$//' -e 's/[^-][^\/]*\//--/g' -e 's/^/   /' -e 's/-/|/'

    find . -type f -mtime -7 -print0 | xargs -0 ls -lt | head
  

    find /var/log/www.lifepim.com -type f -print0 2>/dev/null | xargs -0 grep --color=AUTO -Hn 'Except' 2>/dev/null

    grep -Rnw '.' -e 'blah'

    grep -Rn --include=*.html '.' -e 'blah'
    
    
    

List all processes

    ps -ef
   
   

    pstree
   
   

    ps -u duncan
    
    

    pgrep -a python
    

    top
    htop   (will need to run sudo apt install htop first)
    

Get IP Address and network details

    /sbin/ifconfig

    arp -n   
    
    ip -r neigh
    
    nmap -sA 192.168.1.0/24
    

    nslookup 162.213.1.246
    > Non-authoritative answer:
    > 246.1.213.162.in-addr.arpa      name = wssa.beyondsecurity.com.
  

     host www.acutesoftware.com.au
    > acutesoftware.com.au has address 129.121.30.188
  
  
 Show the routing table

    route 

    nmap

 
 ### Shell tips
 
 Show top commands from your shell history 
 
     history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head
          35 cat
          25 awk
          18 pwd
          15 ls
          14 cd
     
     
  

Download a file

    wget http://www.acutesoftware.com.au/aikif/AIKIF-Overview.jpg
    

    wget --recursive  --page-requisites http://www.acutesoftware.com.au/cont_articles.html

Get a list of URLs from a html file (like an exported list of Chrome bookmarks)

    grep -Eoi '<a [^>]+>' source.html | grep -Eo 'HREF="[^\"]+"' | grep -Eo '(http|https)://[^/"]+' > urls.csv
    

    cat /var/log/www.lifepim.com.access.log  | grep "POST"          # number of posts to lifepim
    cat /var/log/www.lifepim.com.access.log  | grep "login" | wc -l # number of login page accesses
    awk '{print $1}' /var/log/www.lifepim.com.access.log | sort | uniq -c   # count per IP address
    awk '{print $7}' /var/log/www.lifepim.com.access.log | uniq  # list of pages accessed
    
    cat /var/log/www.lifepim.com.error.log | grep "Exception" | uniq      # list of exceptions
    
    awk '{print $11}' /var/log/www.lifepim.com.access.log | sort | uniq -c | grep -v "lifepim"  # count by referrers
    
        1 "https://newsbout.com/id/19184625381"
        1 "https://umumble.com/links/156005/what-software-will-you-trust-when-you-get-senile%3F"
        1 "https://www.producthunt.com/ask/616-what-s-the-best-personal-knowledge-base"

        for i in /var/log/www.lifepim.com.access*.gz
        do
            echo -n "Checking zipped logfile $i - "
            zgrep '/blog/' "$i" | wc -l
        done


        Checking zipped logfile /var/log/www.lifepim.com.access.log.2.gz - 45
        Checking zipped logfile /var/log/www.lifepim.com.access.log.3.gz - 112
        Checking zipped logfile /var/log/www.lifepim.com.access.log.4.gz - 92
        Checking zipped logfile /var/log/www.lifepim.com.access.log.5.gz - 62
        Checking zipped logfile /var/log/www.lifepim.com.access.log.6.gz - 64
        Checking zipped logfile /var/log/www.lifepim.com.access.log.7.gz - 85
        Checking zipped logfile /var/log/www.lifepim.com.access.log.8.gz - 213
        Checking zipped logfile /var/log/www.lifepim.com.access.log.9.gz - 80

Display Annual Calendar for current year

    cal -y 
   
   

    echo $(date -I)
    

    DATE=`date '+%Y-%m-%d %H:%M:%S'`
    echo $DATE
    
   

Show table size of selected tables in a schema

    SELECT table_name as 'Database Name', 
    sum( data_length + index_length ) as 'Size in Bytes', 
    round((sum(data_length + index_length) / 1024 / 1024), 4) as 'Size in MB' 
    FROM information_schema.TABLES where table_name like 'as_%' or table_name like 'sys_%' 
    GROUP BY table_name; 

    select * from information_schema.columns where table_name = 'as_task';

    select DATE_FORMAT(log_date, '%Y-%m'), count(*) from sys_log group by DATE_FORMAT(log_date, '%Y-%m') order by 1;

    select log_date, user_id, count(*) from sys_log group by log_date, user_id order by log_date;

    select WEEK(log_date), max(log_date) as date_until, count(*) as num_user_actions, 
     count(distinct user_id) as active_users_this_week from sys_log 
     where DATE_FORMAT(log_date, '%Y-%m') > '2018-05-05' group by WEEK(log_date) order by 2;

I love writing personal programs - you get to re-invent the wheel your own way to learn exactly how things work, with no code reviews or people shouting "[BLAH] ALREADY DOES THIS".

Sometimes you just want to code and see if you can recreate BLAH for yourself.

My oddest program was called 'heater' - it variable settings from warm -> hot and would peg the CPU at different rates (at the time I was in an unheated office, and rather than go buy a heater - I did what all programmers do and re-invent the wheel)

Programming is fun!

That's great, and I am sure it is cool, but I don't trust Google to keep products maintained any more.

I won't be trying it out.

Well said! Life should be what you want to do - many people don't have ambitions or interests, and I guess defaulting to the evolutionary standard of having kids is a valid response.