ffwd

IMO consciousness is different from working memory, at least to a certain degree. The inner mind (working memory) is different from consciousness from sensory stimuli. You can stand outside and take in the environment and have a relatively quiet inner mind. When you think about math or philosophy a different type of conscious experience arises than sensory stimuli experience. That doesn't mean consciousness is completely isolated from working memory but there is some distinction there I can't describe fully.

Edit: I also think as someone else said, we already know the intermediate layers can contain a lot of adjacent words related to the topic without explicitly outputting those words. These could just be related embedding intermediate vectors that activate but aren't outputted.

Just some speculation, but, I think humans have on the one hand a lot of degrees of freedom in behaviors and thoughts they can do, but at the same time all that freedom is reigned in by our biological needs, like preserving the integrity of our body, but also preserve the integrity of our minds. But this extends further to preserving our surroundings (for our safety, a changing environment brings uncertainty), but also of people we care about and even entire societies that we have. And preserving our future selves through prediction of future environments.

So all that is to say, I'm not sure it is even theoretically possible to create a single algorithm to do open ended search and evaluation. Biology has billions of years of evolution and accumulation, whereas a simple algorithm in a computer, even if smart and connected to the real world, has no such accumulation.

I think humans hit the perfect sweet spot where we have the simplicity of the self preservation instinct, but we have the complexity of the cortex and lots of degrees of freedom because of it, plus on top of that we have a lot of accumulated degrees of freedom in the society and technology and knowledge that have we, which has been built up for thousands of years, all of which we can't just create an algorithm to encapsulate without going through the actual evolution.

And just to make it explicit - a large percentage of what humans think derives from an instinct to preserve the self, the mind, the future and the environment, even if it is very abstract at times. Not absolutely all, but I think a good chunk. And the complexity and degrees of freedom comes from that we have so many neurons in the brain, and a complex body with hands and whatever else that allows a lot of behaviors, as well as a complex environment that is constantly challenging us.

I just want to add that the "recursive" part of recursive self improvement is by no means a given, even if an AI can improve itself.

Recursive self improvement is by its nature a step wise behavior not a continuous one, I would argue. Why? Because you can imagine an AI improve itself by simply fixing random bugs and fixing things using techniques that are in its training, and doing refactoring and so on, all without any real change in capability.

These are not recursive improvements. Recursive improvements usually need conceptual breakthroughs. It is possible to get conceptual breakthroughs with LLMs I believe, maybe it can improve something by tying together ideas from disparate disciplines for example, but I have at least for time being, limited success getting that to work in a way that is creatively new and surprising. Not sure how to get it to feel as creative as the best humans can be.

Well in that case I'm not sure where you're going. I agree that hamsters probably have a similar consciousness to ours, which is kind of the point I was trying to make.

I think that consciousness comes before self-awareness, even though self-awareness is kind of a vague term. Self-awareness can either be an abstract knowledge that you are an organism and a discrete entity in the world (world knowledge/self knowledge), or it can be more basic and be a form of conscious experience, but as my point was, I think conscious experience is broader and does not necessarily need to be about self-awareness.

I'm not sure consciousness and self-awareness are the same thing. First is we can be conscious when we sleep/during REM sleep, where it's arguable we are not self-aware. And if not that, we can even do it when awake, for example when we think about a movie, or a philosophical problem, we can have conscious thoughts that are not related to the self. This leads me to believe consciousness is separate from self-awareness. Self-awareness is _one thing_, among many, that the brain can think about and be conscious of.

Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.

If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.

This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).

Sorry I messed up how I wrote that. I meant to only remove metadata columns, not all columns.

I find that this happens when you enter folders that have media files like audio files, video files and so on. One way to fix it is to enter one such folder, then remove all columns (like file name, date modified - those columns) and remove all the columns that are media metadata columns. Things like track length, artist, contributing artist or whatever else, then click in the File explorer menu on the 3 dots icon (**) and select View tab, then click 'Apply to folders'. This will apply the column and view settings that you just applied to all such folders.

Now all folders with media files open immediately. Also if you want no wait for video files folders, right click in the folder and select 'View -> Details or View -> List or some other option where it doesn't create thumbnails and it'll load even quicker.

I feel like I kind of borked the last paragraph so I want to clarify something.

The point is basically that since these repeating patterns are different every time, they are not emergent. They don't really "exist" except as matter repeating itself in a similar way. Emergence implies there is some kind of different qualitative difference between the emergent level and the lower level but I would argue there isn't.

Sorry for pastebin but it's all I can do atm! Would appreciate your views

Would appreciate your views on this!

Maybe it's looked down upon to complain about downvotes but I have to say I'm a little disappointed that there is a downvote with no accompanying post to explain that vote, especially to a post that is factually correct and nothing obviously wrong with it.

Even though I think it's true that it's lossy, I think there is more going on in an LLM neural net. Namely that when it uses tokens to produce output, you essentially split the text into millions or billions of chunks, each with probability of those chunks. So in essence the LLM can do a form of pattern recognition where the patterns are the chunks and it also enables basic operations on those chunks.

That's why I think you can work iteratively on code and change parts of the code while keeping others, because the code gets chunked and "probabilitized'. It can also do semantic processing and understanding where it can apply knowledge about one topic (like 'swimming') to another topic (like a 'swimming spaceship', it then generates text about what a swimming spaceship would be which is not in the dataset). It chunks it into patterns of probability and then combines them based on probability. I do think this is a lossy process though which sucks.

LLMs _can_ think top-to-bottom but only if you make them think about concrete symbol based problems. Like this one: https://chatgpt.com/s/t_692d55a38e2c8191a942ef2689eb4f5a The prompt I used was "write out the character 'R' in ascii art using exactly 62 # for the R and 91 Q characters to surround it with"

Here it has a top down goal of keeping the exact amount of #'s and Q's and it does keep it in the output. The purpose of this is to make it produce the asciii art in a step by step manner instead of fetching a premade ascii art from training data.

What it does not reason well about always are abstract problems like the doctor example in the post. The real key for reasoning IMO is the ability to decompose the text into a set of components, then apply world model knowledge to those components, then having the ability to manipulate those components based on what they represent.

Humans have an associative memory so when we read a word like "doctor", our brain gathers the world knowledge about that word automatically. It's kind of hard to tell exactly what world knowledge the LLM has vs doesn't have, but it seems like it's doing some kind of segmentation of words, sentences and paragraphs based on the likelihood of those patterns in the training data, and then it can do _some_ manipulation on those patterns based on other likelihood of those patterns. Like for example if there is a lot of text talking about what a doctor is, then that produces a probability distribution about what a doctor is, which it then can use in other prompts relating to doctors. But I have seen this fail before as all of this knowledge is not combined into one world model but rather purely based on the prompt and the probabilities associated with that prompt. It can contradict itself in other words.

That's a good point and I agree. I'm not a neuroscientist but from what I understand the brain has an associative memory so most likely those patterns we create are associatively connected in the brain.

But I think there is a difference between having an associative memory, and having the capacity to _traverse_ that memory in working memory (conscious thinking). While any particular short sequence of thoughts will be associated in memory, we can still overcome that somewhat by thinking for a long time. I can for example iterate on the sequence in my initial post and make it novel by writing down more and more disparate concepts and deleting the concepts that are closely associated. This will in the end create a more novel sequence that is not associated in my brain I think.

I also think there is the trouble of generating and detecting novel patterns. We know for example that it's not just low probability patterns. There are billions of unique low probability sequences of patterns that have no inherent meaning, so uniqueness itself is not enough to detect them. So how does the brain decide that something is interesting? I do not know.

I think something that's missing from AI is the ability humans have to combine and think about ANY sequence of patterns as much as we want. A simple example is say I think about a sequence of "banana - car - dog - house". I can if I want to in my mind, replace car with tree, then replace tree with rainbow, then replace rainbow with something else, etc... I can sit and think about random nonsense for as long as I want and create these endless sequences of thoughts.

Now I think when we're trying to reason about a practical problem or whatever, maybe we are doing pattern recognition via probability and so on, and for a lot of things it works OK to just do pattern recognition, for AI as well.

But I'm not sure that pattern recognition and probability works for creating novel interesting ideas all of the time, and I think that humans can create these endless sequences, we stumble upon ideas that are good, whereas an AI can only see the patterns that are in its data. If it can create a pattern that is not in the data and then recognize that pattern as novel or interesting in some way, it would still lack the flexibility of humans I think, but it would be interesting nevertheless.

Yeah interesting I have to think about this.

No, the new algorithms used to be determine this was created by ICM-CSIC who are also the publishers of this article.

Also the authors of the paper is involved with the article, there is for example this quote:

“We are witnessing a true reversal of ocean circulation in the Southern Hemisphere—something we’ve never seen before,” explains Antonio Turiel, ICM-CSIC researcher and co-author of the study.

Yep definitely agree with this.

Why can't it be algorithmic? If the brain uses the same process on all information, then that is an algorithmic process. There is some evidence that it does do the same process to do things like consolidating information, processing the "world model" and so on.

Some processes are undoubtedly learned from experience but considering people seem to think many of the same things and are similar in many ways it remains to be seen whether the most important parts are learned rather than innate from birth.

I think humans have some kind of algorithm for deciding what's true and consolidating information. What that is I don't know.

This to me is the paradox of modern LLMs, in that it doesn't represent the underlying domain directly, but it can represent whatever information can be presented in text. So it does represent _some_ information but it is not always clear what it is or how.

The embedding space can represent relationships between words, sentences and paragraphs, and since those things can encode information about the underlying domain, you can query those relationships with text and get reasonable responses. The problem is it's not always clear what is being represented in those relationships as text is a messy encoding scheme.

But another weakness is that as you say it is generative, and in order to make it generative we are instead of hardcoding in a database all possible questions and all possible answers, we offload some of the data to an algorithm (next token prediction) in order to get the possibility of an imprecise probabilistic question/prompt (which is useful because then you can ask anything).

But the problem is no single algorithm can ever encode all possible answers to all possible questions in a domain accurate way and so you lose some precision in the information. Or at least this is how I see LLMs atm.

But the problem is we don't know how it works. It's not about assuming consciousness is outside of physical reality or something like this, it's simply the fact that we don't have an understanding of it.

For example if we could see and trace all intentional thoughts/acts before they occurred (in matter), intentionality would cease to be a property, it would be an illusion.

All things that we know of in the universe function as physical matter, and we know the brain is a physical thing with 80 billion neurons and trillions of connections. What's the simplest explanation?

1) This is an incredibly complicated physical thing that we don't understand yet (and quite naturally so, with it having an incredible number of "moving parts")

or 2) there are qualitative elements in the universe that we don't have the scientific tools to measure or analyze, even in principle

I go with #1 because that's what every fiber is telling me (although I admit I don't know, of course). And with #1 also comes reductionism. It is a physical system we just don't have the mental models to understand it.

I also want to say there could be another aspect that affects consciousness - namely the appearance of a "present now" that we experience in consciousness. This present moment is not really explained in physics but it could have something to do with how consciousness works. How I don't know but it all relates to how we model physics itself mentally.

This could be true but it depends on the person. Marc Andreesen is a billionaire and he wears Rolexes quite prominently. In fact a lot of rich people wear Rolexes lately.

To untutored common sense, the natural world is filled with irreducibly different kinds of objects and qualities: people; dogs and cats; trees and flowers; rocks, dirt, and water; colors, odors, sounds; heat and cold; meanings and purposes.

It's too early to declare that there are irreducible things in the universe. All of those things mentioned are created in the brain and we don't know how the brain works, or consciousness. We can't declare victory on a topic we don't fully understand. It's also a dubious notion to say things are irreducible when it's quite clear all of those things come from a single place (the brain), of which we don't have a clear understanding.

We know some things like the brain and the nervous system operate at a certain macro level in the universe, and so all it observes are ensembles of macro states, it doesn't observe the universe at the micro level, it's then quite natural that all the knowledge and theories it develops are on this macro scopic / ensemble level imo. The mystery of this is still unsolved.

Also regarding the physics itself, we know that due to the laws of physics, the universe tends to cluster physical matter together into bigger objects, like planets, birds, whatever. But those objects could be described as repeating patterns in the physical matter, and that this repeating nature causes them to behave as if they do have a purpose. The purpose is in the repetition. This is totally inline with reductionism.

The problem for me is that we haven't conceptualized what we mean by "reduced" consciousness. Reduced in what way?

If we create an analogy with audio - audio can have 2 properties: frequencies and volume. Frequencies are the content of the audio and volume is how "present" the sound is.

Well the same could be applied to consciousness. When we say reduced consciousness do we mean that the mind experiences less content (frequencies) or do we mean that all the frequencies are there but at a reduced volume?

This is an interesting idea but personally I think the opposite - the universe is not lazy and all details matter at all levels.

Like imagine making a complete account of all world views of all people in all of history - all perspectives, and all the physical events of that history. There is almost infinite detail there. In a way, in the universe all the details of all the things matter, including at the physical level, otherwise you wouldn't get the diversity and complexity you get now.

This is good advice but just a note - the point is to be as simple as possible. With SSL and a good password there is no hacking that I know of. If I started doing a lot of this the file would get bloated and the readability and extensibility of the code would go drastically down IMO.

I agree and I think if there are any hazards to modern information landscape it's kind of 2 fold (that I can think of) 1) the brain likes simple generalized information/models more than specific information and 2) the brain likes information that confirms our existing beliefs more than staying in a gray area where it's not sure what information is true or not.

Regarding first one I think one of the main challenges will be finding which general explanations are true or not. This can be anything from cultural things like gender discussions to conspiracy theories to scientific theories. If there are generalized models that explain a lot of phenomena, the brain likes those, but at the same time they can also take the place for when empirical evidence should have been gathered (that doesn't fit with the general model), and so in that sense they can be hazardous. And models that already have stood up to a lot of empirical data are especially hazardous because they have already proven themselves to some extent which means there's even less incentive to gather empirical data. Or at least that's my current (maybe hazardous) general view.

I also don't think we should give up on generalized models because they are so powerful and useful, but it's definitely a challenge in todays climate because they get so many clicks and they are very satisfying to know.

There are two types of gradient though, conceptually. If consciousness is some state of matter that is unknown still, and each neuron, for example, contains "one bit" of consciousness, then the gradient is that as you add more neurons, you add more complexity to the consciousness, but you do not change the fundamental experience of consciousness. You add more content but not more experience in itself.

If on the other hand consciousness is this emergent phenomenon that depends on neurons and their connections, then the gradient (and thus the experience) would be far more diverse and there would be a lot of different ways consciousness could "feel".

The problem I have is that for example, as far as my brain can remember, stimuli has looked the exact same all throughout my life. If I saw my a tree when I was 10, and I saw the same tree now, the conscious "qualia" of this would look exactly the same. To me this is a mystery, that the connections in the brain do not change the experience of qualia at all. Red looks like red no matter what the neuronal state of your brain is. I don't have an answer to this but just something I've been thinking about.