Modelling creativity

[Spenner]

I've been thinking of what might be involved for a computer program that reveals a form of creativity, given that it has tools to analyze things around it and constantly re-associate with previously learned knowledge.

If one were to program in some fundamental rules, such as composition, shape and contrast, understanding focal points and how one might be interpreting the elements when seeing it (to a computer, generated static is definitely an interesting picture to analyze; however to a human, we will not spend time picking up patterns of how such small pixel patterns might have been generated). A computer model like that would need to understand broader, more specific, modular elements that do not have to do with the fundamental processes that generate the pixels on a screen. It needs to look at the bigger relationships at play; how does shape A work with shape B in the context of what's on the screen? What theme can be illustrated?

However I think the bigger problem would be to try to convey to a computer exactly what things such as a "theme" and a "pretty picture" might be. Logically, it could create something which would be the bare minimum of what might be "compositionally correct" or the bare minimum for what a "colour-cohesive image" might be. A challenge would be to not let it stop at once analysis, but to constantly analyze what it's done previously, to find mistakes, to build more relationships with the other shapes it can compute.

What would those relationships be? On a very fine level it might be something as simple as "this grouping of pixels must not have a dense packing of pixels within a 100 pixel radius of the surrounding shapes", which could be condensed into a modular packet called "spacing". To get a computer to not choose the value of spacing needed for each drawing randomly might be tricky, because it needs an influence and reason for choosing that value; this could only be established by having the program analyze a large number of artworks and trying to compute why certain things work in each drawings, and projecting those ideas on the screen.

All this being said, it's not exactly creativity as much as it is a logical, straightforward approach to generating art. The trick would be to break the logical barriers and to influence trying things that are outside of the box, by having possibility generated from the imagery it's seen. A creative outcome of having it analyze paintings and finding out what aesthetics have been employed for each, would be to then compute a painting on screen that is an "average" of all techniques used. Something pretty abstract that would be hard for a human to decipher, because our individual minds influence what's good about a drawing, there is no objective reason for a colour choice being good for most people. To a computer, if it were to decide that, since a lot of paintings it's looked at were warm in colour tone, deep in contrast, and sharp in edges of defined shapes, that such characteristic looks are what makes a good painting (trusting that we show it examples of what collectively are considered very good pieces), it should be able to project those ideas in an original piece.

A missing point in all that would be choosing what to illustrate conceptually. A very technical projection of these rules might as well be something jumbled, messy and distorted, that has to significance to the human eye, and is unrecognizable. A computer model's "decided creativity" and a human's idea of creativity conflict dangerously because of what appears to the computer as merely an abstract view of something we humans see very clearly defined. It would be very important to have a strong grounding in shape recognition and being able to relate those shapes to an inventory of human and english concepts. Having an english dictionary plugged in, being able to respond back in a human-like fashion, would be key. That sounds like it might just need a rich database full of examples. Unless we can find some sort of algorithm for thinking and seeing like a human does (which frankly is a very dummed down version of the potential of what a computer might be able to observe given the right tools-- imagine programming a computer program that is as good at math as a first grader, and as good at english as one, instead of being perfectly instinctively clear with every computation). Being able to fill in the gaps, ask those extra questions that come out of abstractions, might be what separates a calculator from a more organic, human-like minded calculation. A robot that has to draw out it's work, from step to step, occasionally misinterpreting things because instead of an inherent value of 1 or 2, it is instead working on the associated shapes of those numbers, and the context it has on the paper, it's surrounding shapes, and many other countless variables.

I suppose I just wanted to rant about something I was thinking about, but I'd like to know what other people think about the potential of having any form of generated creativity. I'm far from implying that I've covered all of the necessary grounds here (I haven't even touched it to the slightest degree @_@) especially things like looping self-referential systems that are going on which are constantly becoming more complex, blahblah I won't let myself dig a deeper hole for my uneducated thinking. Just a curious rambling. What say you?


EDIT: I'll add on a few extra thoughts I was having aswell. I was on the back deck of the house I'm saying at, looking at a small fluff ball floating around, but in between the pickets of the rail of the deck. I was automatically seeing how one space in between the pickets related to the next one, not in one continuous sequence, but in little packets. I'll make a quick lil image:



Now, in that example, the next portion of line isn't necessarily related to the previous one. However, there have been variables of physics in the surrounding environment that have been applied to the previous frame to get the feather's path to shift to how it is illustrated in the next.



In this example, it's using the first frame (1) and constructing an idea based on given shape, but also based on a bit of a random influence of chance. Say that it has a set of rules that prevent the lines of the drawing from extending past more than an inch more than the original diameter of the circle (for example, just some kind of aesthetic rule).

A random decision to draw more lines off the drawing in (2) will illustrate no more relation to (1) than that simple rule. However, in (3), it has assessed the difference in characteristics between (1) and (2) and seen that the straightness of the corner in one of the little legs is out of character with the curvature of the rest of the drawing, making it out of context. It will then solve the context by erasing that limb and giving it curvature which is cohesive with the rest. The same thing happens in (4) and (5), though it's not really as accurate because after the first mistake in (2), generally, the rule should apply for the next set of drawn elements.

Just a very very small way that a computer could associate and build aesthetic cohesion with a very abstract "theme". The complexity of the drawing would be initially decided when envisioning the relative size of the ideal end image.

It wouldn't be an objective rule after computed for this drawing-- each decision will be individual from the last, with new parameters chosen given what's already been done. Perhaps a menacing face makes it's way onto this little crab in frame (6), making the limbs feel out of context, so in (7) the computer wishes to change all of the elbows of the crab's limbs to sharper corners. This wouldn't really be the case in my example though, because it's already been established that there is curvature in the general theme, so a happy face (with another element of curvature in the mouth) would feel far more cohesive.

[UserNameGoesHere]

I wouldn't call it "modelling creativity", but other than that you have the general idea correct for what would be a type of artificial intelligence program which could evaluate drawings or paintings and tell roughly how "good" they are and possibly try to emulate some of the "good" styles.

I think a better thing to call it would be "emulating art styles with artificial intelligence". The reason I wouldn't call it anything "creativity" is because firstly one must define what one means by "creativity" and, secondly, because it doesn't even remotely fit my idea of what "creativity" is. I would say trying to emulate someone else's style is the very opposite of what creativity is.

[Spenner]

True, but I suppose the idea is to find influences from elements it analyzes, in an attempt to "inspire" the end product the machine would produce, rather than to just allow it to copy concepts from each drawing and to directly copy them.

Creativity is a pretty subjective definition though, there's no objective measurement that can be made on it, so it's not really a tangible thing to reproduce in the full warmth as it appears with humans. I think "modelling creativity" could only be relevant if you think of it as a model for merely appearing like creativity, by taking inspiration from a certain set of analyses and to be able to create a new composition.

It's either a system of analysis or hard programming these inherent styles and inspirations to choose from at the start. However I think having a system that is constantly updating its database of references and comparatively battling out what, based on criteria observed, is the "best" set of traits to bring forward.

I guess in a nutshell it's "survival of the fittest" in terms of what stylistic traits stay and which go. Once you realize that choppy lines don't look as good as straight lines, you will adhere to this new found better way of illustrating, and not look back. However, it's never deleting anything. Certain styles are relevant in certain contexts-- if the computer is asked to, for example, draw something in the style of a child's style, it needs to have elements which illustrate associations of "children" and "imprecise" and "sloppy" (perhaps) to best represent the style it's been given to reference.

I think that if really polished and rich with a database it could be something like a Cleverbot that draws things instead of just responding. Cleverbot is cool but it really REALLY lacks a human influence because of its lack of learning capabilities. It can't refer to subjects of the past in a cohesive way that makes it's decisions of what to talk about seem human-like. It's always robotic.

In the same way that one would expect a true Cleverbot-like system to speak coherently like a human might, this ARTificial intelligence (perhaps a trademark name as ART, Official Intellgence??? jk thats too corny) should express visual perceptions like a human, and draw in a way that projects what a human would want to use in a drawing. An organic set of reference points for learning what to draw. okay rambling ovah

EDIT: Just a thought on the subjectivity of this,

Originally Posted by UserNameGoesHere

I would say trying to emulate someone else's style is the very opposite of what creativity is.

I entirely disagree, because emulating someone's style is what gains insight into new styles of your own. Take pencil drawing for example, the method of shading used and ideas about where the darkest shadows go/what style of line work best emulates realism probably SHOULD be copied, if wanting to employ realism. Copy until you understand how that kind of style shapes the final idea. Afterwards you can critique your own work and think about how you can personalize those skills you've learned. A "style", I think, is just a fundamental way of expressing an idea, the types of rules you adhere to as you're expressing it. Lines should be thick, lines should be soft and scattery, shakey and scratchy, pointillism, etc.

In a nutshell, it's how you execute the idea is up to you-- there's your own style you've gained which is a result of trial and error, or being inspired by seeing good examples of artwork around you (for me for example, I love salvador dali's type of shading, and how it shapes a realistic looking shape which is typically a surreal element. I am in no way emulating him other than a small aspect of his style, and I don't think that renders my own artwork uncreative because of elements present in other drawings I was inspired from). With that logic it would be like saying a mathematical solution isn't creative because someone else has developed trigonometry before you, and it's not your true solution. Which really isn't true because it's how you've used those concepts to solve a problem of your own. It's the same thing here with using your stylistic tools to personalize whatever you want to make.

That being said I think this machine could be considered a bit more creative after being able to resolve that idea.

[UserNameGoesHere]

You see how it depends entirely on how one defines "creativity" though.

To me, creativity has more to do with finding new and novel approaches, rather than emulating or reworking old ones. Sure, a reworking of an old style which includes some novel component is creative, but only in so much as the novel component is, and even then it's not as creative as, say, a completely novel style.

As far as mathematics, I would argue that mathematics itself is not creative, although the approaches to it can be. But what is the essential mathematics itself essentially pre-exists and counts as a discovery rather than an invention, although mathematical notation counts as an invention, but I hope you see the difference between mathematics and mathematical notation.

Copying good styles is good and fine, but it's not any more creative than copying an answer off someone's test is "knowing how to work that problem".

[Spenner]

Again I think the difference is where one is finding the approach to the solution. Copying someone's answer isn't creative past the ability to find a way to cheat creatively.

I also think it depends on what mathematics we're talking about. To a certain degree, all of mathematics are a pretty creative concept. A symbolic string such as "1 + 1 = 2" is such an ingrained ability for us, but to assign values and expressions to the symbols, to be able to understand the chemistry between them, and how they build a new product symbol, is a pretty creative trait. That's a pretty crude example, but when you take into account application such as mandelbrot set fractals, crazy geometry and things of the like, to be able to perceive the logic going on under the surface involves a creative mind that's able to adhere all of the abstract symbolism going on.

I for one am very poor at mental mathematics, when I try mental math I begin to visualize the symbols of the function or problem mending and blending in a strange fashion. I don't really perceive the values which are attributed to the symbols under the surface. It's easy to take for granted the ability to do that kind of abstract problem solving. I'm much better at being able to visualize a function as a graph or fractal or something like that.

Copying a style and being able to understand what the fundamental concepts behind the style are, being able to apply your own ideas through that system of aesthetics, and to be able to express it, is a creative task IMO. One could argue copying a fantastic pencil drawing to try to make an exact replica takes a really effective amount of technical and creative dexterit/y, to be able to replicate the same expressive style that the original artist had given it, especially if highly stylized. I think that we take for granted the amount of creative problem solving going on in the brain as we attempt to put something in our own hands, whether it's copying a style or solving a math problem.

But you're right, the nature of what creativity is happens to be an issue faced here. PS I really don't mind if this is a thread debating what creativity might be or what it might involve-- I love thinking about what really makes something creative or not. Sometimes it involves what kind of an argument you can make for it.

I think "creativity" is almost too spacey of a definition for it. Creativity in the way I see it is the ability to problem solve at a fundamental, unbiased thinking level, and being able to grow those ideas.

I don't see how creativity is any different than problem solving, it is just a word with a stigma tied to more intriguing, out of the box type solutions. But they really are just solutions to abstract or quite literal problems. Creativity in a rich human context involves associations with emotions, letting certain algorithms of the mind play out on auto-pilot, really abstractly looking at things, and being able to continue to build those abstractions. Inspiring a program to do something that rich would involve a lot of hardware, and probably would need to involve a quantum computer to process things at the speed required.

[UserNameGoesHere]

Again, one must distinguish between mathematics and approaches to mathematics, such as mathematical representation, including notation. The intrinsic mathematics itself is pre-existing. Things have height, width, weight, and other mathematical properties, independent of human perception. Further, the mathematical relationships between things are also pre-existing. This sort of mathematical order of the universe pre-exists. What humans do is try to understand this via representation or notation or via different approaches -- the approaches themselves which may be creative, but the underlying mathematics pre-exists and is discovered rather than created.

It's a fine point, but it's basically the difference between the symbols and notations that we use (which are invented), and the underlying mathematics itself (which pre-exists independently of humans). Certainly looking at a problem in various ways is creative, although the problem itself (the intrinsic mathematics) pre-exists. It is entirely possible and in fact likely for human understanding of mathematics to be lacking -- however the underlying mathematics is perfect and pre-existing -- it exists whether we understand or comprehend all of it or not as it is essentially the "rules of the universe", quite literally.

You mention being able to understand concepts so that you could apply your own ideas being a creative task. That may be, but I would argue this cannot apply to computers because they cannot understand concepts nor do they have ideas. So while the end result may be similar, if there is no intention behind it, it cannot be creative. And since a computer has no intention (it blindly obeys without understanding or meaning), how can it possibly be creative?

Now, the person who made the program could be creative. The people who made the art it learned from could be creative. The program itself could not be. That is my thinking.

[Spenner]

I'm totally with you on the fact that mathematical laws are as objective as the laws of physics-- their natural properties are inherently concrete, it's just a matter of discovering more of it.

What I was getting at, is that to apply a language to describe and articulate the syntax of the mathematics is the creative part, the utilization of an analogy for the mathematical world. The mathematics themselves are there whether we are aware of them or not, and I'm sure countless more systems of mathematical logic are yet to be found or examined in a more accurate way.

All of the fundamentals are independent of human perception, but it's the perceptual tools that understand them and condense these things into organized syntax that is the creative part. "1 + 1 = 0" could just as easily be expressed as "+ 1 + 2 T". If you had something different, you'd indeed NEED to have a creative insight on how to interpret the patterns of symbols present. But, again, the values of the symbols are unchanged.

About a computed idea... well, what is an idea, once broken down? There are a vast populous of different kinds of ideas-- the idea to hold your breath, based on being submerged in water; the idea to walk because the ground is ideal for walking, and because you have an idea of balance integrated into your body still, and can carry that into a walking motion-- it really all depends on the context. It can be as simple as the idea to choose red over green, or as elaborate as the idea for finding the shortest path of a 3D maze of which consists of 6x10^5 alternate routes.

It's interesting because for a computer, the second task might actually be simple, and doesn't even require a so called "idea"; it's just deducing using simple logic. But for a human, finding the solution through hard labour of working it out is a creative task of idea generation. For a computer, however, choosing red over green, with no additional parameters attached? No extra conditions? That would be difficult, given that there's nothing for it to go on. A computer has no favourite colour, they do not play favourites on their own.

You're right that a program itself could not exactly be creative if it was only through the mind of it's coder, a non-expansive projection of the code that is written. But that is not what I have in mind-- the programmer would be building the rules in which it will take in new information, give it the system of logic for which it will use to sort out patterns that adhere to what it will collectively deduce as "good" or "bad" examples of certain styles of art. It wouldn't be an objective good or bad for a computer, it would simply be different styles, and therefore go into different categories. A child's scribblings just categorize as artwork represented by a child's abilities.

Whether something that can appear to have a creative approach works or not, I don't think it would be anything more than a database organizer, and selective pattern finder, given conditions to work under. My hopes would be that said databases will be constantly comparing one another, in a way that acts like parts of the fundamental parts of a brainstorming train of thought, a creative flow, might reflect. It might not be a true representation of it because of the restriction that it is not human. However I think there can be a machine built that would be able to have the traits which reflect what creative idea generation might consist of without adhering a personality to it that has a bias for certain things.

A tricky point is that all people draw certain things because they have a strong feeling for certain styles, which goes into deep connections emotionally or with things they aspire to be able to create. For a program or machine like this to work it might need a suggestion-- perhaps it would work like Google, using a string or question to search through it's database and construct a result which is the most accurately tied to your search string.

A BIG vision would be this: having a profile which you give some personal insights into your life, your moods, the things you think about a lot, favourite colours, this and that; a questionnaire to stick to a profile which, when searching or asking the program to draw something, would use it as a reference point of the kinds of influences to have on it's final product. Overall I just think it would be a really amazing tool to be able to check off certain things like "style: hand painted, pointillism, impressionism, abstraction, even Surrealism", colour schemes, and the content of the drawing, and have something generate it. Something brand new, something that creates concept art from nothing but suggestions and from a rich database of references.

-----

Realistically I think something like a vector based output would be the most tangible to make. Think the kinds of drawings you'd make in Illustrator. Not to mention, it's most likely that a vector system would be used to map out and collect data from artwork around the web (or manually inserted), mapping shapes for references, collecting spatial recognition of certain elements, vector directions for a lighting scheme, many many other details I think would work well with a system like that.

A prototype would very much need to be vector based for analysis IMO but I think such a program could then have a rendering system that generates into a raster image, building detail with pixels that again are adhering from analytical observations, but this time are the more detailed observations such as: texture, edge detection/edge softness, shadows and highlights, details and definition of subtle shading, characterizing details (wrinkles perhaps, cracks, etc).

These are all very big ideas and DAMN I am rambling so I'll cut myself off there

[UserNameGoesHere]

Again, what you are describing is totally doable. It would require quite a lot of work but it could be done and it falls perfectly into the realm of artificial intelligence, as I've stated. It even could come up with some very interesting results which, if produced by a human or other sentient being would possibly qualify as creativity.

My only contention is that since the machine has no intentionality (for it has no will, intent, nor understanding) that it therefore does not possess creativity.

Similar arguments could be made asking if artificial intelligence could ever truly become intelligent. I would say no. Most AI researchers would say no. A minority of AI researchers would say maybe or yes.

But what it really comes down to is symantics. And if we are talking about two different things but we call them the same word, they are still two different things, even if they have similarities.

So a lot of it comes down to how you define "creativity", how you define "intelligence", and various other things like that.

Also, it is very easy for a computer program to distinguish between red and green, yet it would be very difficult (in fact, NP-Complete difficult -- see Travelling Salesman problem, for example) to pick the absolute shortest of 6x10^5 possible routes.

So I don't know where you got the idea that the color pick would be the more difficult for a program to do, out of those two tasks.

[Spenner]

It would be easy for it to decide a colour at random; but the point is that a computer won't have a favourite colour, so it needs to decide red or green based on certain conditions which aren't ingrained in it. It could give you a precomputed response but much like a machine wouldn't possess consciousness it wouldn't have a personal choice in the matter of whether or not it likes red or green.

Another thing I'm pondering-- at what point is our brain not machine-like? At the synaptic level I'd argue that there's potential to replicate those systems of organizing and acting receptively with signals. Mind you that would take a lot of small components in order to model something so immense.

Also about finding the shortest path, what about an algorithm that's just trying out all of the paths until it finds the shortest one? Something like a quantum computer could do that very efficiently, but of course we're not able to do that right now.

I think there's forever going to be the barrier of artificial intelligence being just a reflection of what appears to be traits of an intelligent mind, which is just a series of precise calculations to the machine. We could make something believable but, at the core, still just a machine.

[UserNameGoesHere]

Originally Posted by Spenner

It would be easy for it to decide a colour at random; but the point is that a computer won't have a favourite colour, so it needs to decide red or green based on certain conditions which aren't ingrained in it. It could give you a precomputed response but much like a machine wouldn't possess consciousness it wouldn't have a personal choice in the matter of whether or not it likes red or green.

Well yeah it wouldn't have a favorite color, but if you were to ask it, say, which of these colors is red and which is green, a program could easily distinguish between the two.

Another thing I'm pondering-- at what point is our brain not machine-like? At the synaptic level I'd argue that there's potential to replicate those systems of organizing and acting receptively with signals. Mind you that would take a lot of small components in order to model something so immense.

Neuroscience still doesn't 100% understand the brain. I think we first need to fully understand the brain before we can even think about trying to replicate similar functionality in any reasonably accurate manner.

Also about finding the shortest path, what about an algorithm that's just trying out all of the paths until it finds the shortest one? Something like a quantum computer could do that very efficiently, but of course we're not able to do that right now.

That's just called the brute force algorithm. It's one of the least efficient algorithms and won't work on many problems whereas other algorithms will, simply because even with modern computing power, a brute force approach to some problems would still take hundreds of years, thousands of years, or longer.

As far as I know there do not exist any quantum computers. There have been some which have been claimed to be quantum computers but which have not been proven to be so. So quantum computation is strictly theoretical unless an actual quantum computer were to exist at some point.

I think there's forever going to be the barrier of artificial intelligence being just a reflection of what appears to be traits of an intelligent mind, which is just a series of precise calculations to the machine. We could make something believable but, at the core, still just a machine.

Yes.

Also, I think you would be interested in, would benefit from, and should read a good book on Artificial Intelligence. It should cover most of what we've been discussing and you should see different views on things, things which have been tried, etc... Unless you're really interested in the programming side of things, try to get a book which isn't so focused on programming but more just an overview of AI.

https://mitpress.mit.edu/books/artificial-minds
^Here is one such book you may benefit from reading.

If you have a university with a science library near you (I mean a dedicated science library -- not just a library which also includes science texts), you could go there and browse through their offerings in the computer science section.

[Cavernio]

We could create a computer program that has likes and dislikes, (irrespective of there being a mind,) and it wouldn't be all that different from how people get likes and dislikes. Let's consider some known reasons why people like things. 1. Programmed to. Eg: Babies like sugar, dislike bitter. 2. We like things we encounter more than new things, generally. (Faces, foods, music to an extent...has been proven experimentally in stripped down, lab settings.) 3. Other
We have 1 down...assign it a 'taste' directly based on someone else's preferences or give it innate, basic tastes for certain colors.


For visual art, as has been touched on, you'd probably need to define art to the program by having it break down art into basic components. Lines, shapes, color, location are basic ones. More complex ones could involve object recognition and relative placement of any other element to another one....not sure if these would be necessary as, given enough time and a proper determination and visualization of the 'basic' components, a fully-functional version of this sort of program would create it's own secondary components, or they'd all end up being accounted for within the basic components...but that seems far-fetched to me, at this point.
3 is a random element.
Tying it all together is, obviously, the highly challenging part. And with little programming expertise, I'm not the one to get into this all. But I'm imagining something like a civilization battle in terms of possible outcome. For a computer to draw something, each area of a picture has a chance to be specific color(s) and a specific shape. Assign values for the innate preferences of each pre-existing element, say 1 for utter dislike and, oh, we'd probably need something fairly high, 1 million for adoration. These values then influence a randomly generated number for each area of the picture. Simple, basic example of this sort of idea: Red is a value of 2, white is 1. Add it up, you get 3, so we'll generate a random number between 1 and 3 inclusive. The outcome will be red for 2/3 of the numbers (doesn't matter which numbers, matters how many numbers), and white for 1/3 of them.

In order for the program to get preferences based on experience, you simply show it pictures. Whatever elements it sees in those pictures would then raise the value of the number for that element, (not necessarily in the same area even) thereby making it more likely to happen when the program would create something.

I suppose the most basic, basic level of this IS simply color and area...lines and shapes are secondary to that. You could support getting shapes and lines by initially programming it such that adjacent colors influence the current color. You could create lines instead of swatches of color by making it look at thickness of the entirety of already created painting, and have a strong preference for continuing a line. Same goes for a shape, with a very, very strong influence to 'finish' a shape of a specific size.

When 'painting', the program would work best to paint continuous swathes, or spiraling outwards or something. Or maybe create 4 or 5 'spots' and then work out from each of them.

I guess the biggest drawback to such a design is really making those secondary elements. What I'm describing, even if it saw a million cars, seems like it would only end up making a 'car' on it's own if it were exceedingly similar to the cars it's already seen. But maybe not, maybe it'd just be really, really fine balancing act of how weighted the likelihood of each element would be.

In any case, creating a program that would mimic something rather creatively and with specific tastes, wouldn't be too hard. Making it make more than abstract art feels really hard.


edit; Wow this post isn't as organized as I thought. Oh well.

[Cavernio]

This thread immediately brought to mind this.
http://sunsite.univie.ac.at/Mozart/dice/
The discussion's been about visual art, but music seemingly has a lot more ingrained rights and wrongs, (music theory is quite strict), which makes it a lot easier for a machine to mimic.
I haven't played around with that tool yet, but I learned about it in a music class I took years ago.

[Spenner]

Originally Posted by Cavernio

We could create a computer program that has likes and dislikes, (irrespective of there being a mind,) and it wouldn't be all that different from how people get likes and dislikes. Let's consider some known reasons why people like things. 1. Programmed to. Eg: Babies like sugar, dislike bitter. 2. We like things we encounter more than new things, generally. (Faces, foods, music to an extent...has been proven experimentally in stripped down, lab settings.) 3. Other
We have 1 down...assign it a 'taste' directly based on someone else's preferences or give it innate, basic tastes for certain colors.

I think that you can really only look at programmed preferences in total infancy; later on the reasons for tastes are far more complex and based on a palette of experimentation/trial and error/associations with experience and whatnot. In the same way which a child or baby would have to try many things in order to get a strong understanding of what it likes and dislikes (much like adults develop into eventually), I think that a machine that distinguishes taste on trial and error is necessary.

I think that a machine would have to LEARN what is good and bad and using the patterns/knowledge of someone's preference profile, it can then model those parameters forward into the next set of works. For example, the machine accomplishes it's first drawing. It depicts a lot of mistakes. The program then feeds out a survey to ask a group of people to evaluate the drawing based on personal preference; something like this, I think, would be an effective way to shape a "standard" preference profile.

Having each piece be evaluated by humans, over and over again, and by the program learning from it's mistakes each time, tweaking over and over again until it gets closer and closer to a perfect score on the evaluations. The idea thenceforth would be that the computer can judge itself on a basis of knowing what is expected, generally, through all spectrums of subjective taste (assuming there is a variety of questions that are subjectively based, not just objective things such as "is the linework clean").

All in all I think criticism and trial and error is expected with a robot or program that's expected to interact cohesively in the real world without having everything pre-programmed in. A robot that sees an obstacle might not be preprogrammed to avoid that specific shaped obstacle, that would be silly. Instead it needs to learn to understand that it must avoid solid objects, and that general rule will be ingrained in it's programming after it's first experience with colliding with an object of the sort. The trick would be not to generalize this with ALL objects, and to keep a profile of the details of that specific object (if it were told to avoid all obstacles all the time regardless of context, it might be unable to move past a series of sponges set out in a line in front of it. Through claws, testing its surroundings, it can learn what is achievable with it's hardware).

Originally Posted by Cavernio

For visual art, as has been touched on, you'd probably need to define art to the program by having it break down art into basic components. Lines, shapes, color, location are basic ones. More complex ones could involve object recognition and relative placement of any other element to another one....not sure if these would be necessary as, given enough time and a proper determination and visualization of the 'basic' components, a fully-functional version of this sort of program would create it's own secondary components, or they'd all end up being accounted for within the basic components...but that seems far-fetched to me, at this point.
3 is a random element.
Tying it all together is, obviously, the highly challenging part. And with little programming expertise, I'm not the one to get into this all. But I'm imagining something like a civilization battle in terms of possible outcome. For a computer to draw something, each area of a picture has a chance to be specific color(s) and a specific shape.

Yes you would definitely need a system for breaking down not only the art it observes, but the art it creates, as it's creating it. Ingraining the program richly with both human and self-evaluative rules will help the program thrive. The program could even be learned to question the decisions it's made once they've been made, and could reach out to someone and ask questions. "Is the element in region A working with the element in region B?" and a human can answer it, so that the computer can learn to understand spacing and how it works. Now, this is entirely subjective, and specific to every piece of work. So it will ultimately be decided by the program to make a choice, or to at least propose a choice, criticize it, decide whether it's the most appropriate choice based on what other parameters it affects (if I make this blue, will the contrast between the red object next to it be too great?).

There's an infinitesimal list of variables at play here, definitely not a task that can easily be grasped. But I know there are robots being built that do just this: observe, associate, and recognize, and relate back when necessary. This could definitely be elaborated on within a computer program. In a reiterating, self referential fashion, it will be a constant dance with associating context of elements and on what more can be built. After a lot of "training" the program to learn, and building a database of info and densely packed variables, it can be more independent. Much like teaching a child how to learn to draw, the program would need to adapt the curious and observational framework of an infant mind. Something highly less structured as a real mind of course, but something that models the dynamic workings of a learning machine.

That being said, I think that off the bat, the elements in a piece will indeed be at random. Hell, it might as well have it's first piece of visual art as a generated static noise. Applying it's database of knowledge it should be able to recognize what can be done with the noise-- what shapes can be built and what spots can be altered to compose something, and once it works for a few iterations, it looks at what it's done, decides if it should be changed, and then builds some more in spots not yet affected.

The whole system would be very algorithmic, very reiterated in that the same algorithms used to observe (based on the variables at play for the given observation, i.e. checking for curved lines) are constantly working within themselves, and interacting with other algorithmic systems like context (a HUGE packet of tools connected to databases, working all the time to see if the subject matter seems to relate to the patterns and rules it's learned along the way), so that it can work to make curvature in lines, but only if they are appropriate.

Originally Posted by Cavernio

When 'painting', the program would work best to paint continuous swathes, or spiraling outwards or something. Or maybe create 4 or 5 'spots' and then work out from each of them.

A decent idea, I think there are many number of ways the program could begin something. Perhaps that can be something at random. It would be a good starting point for it to start generating associations; start with a circle, circle reminds of apple, so it draws a leaf on it-- that reminds it trees, so it builds a tree, then it recognizes that there are more than one apples on a tree, so it draws more, then recognizes that a tree sits on the ground, etc etc etc until voila, a logical system of associations has built something for us. It really doesn't require a human mind at all per se, and it's probably the same logic google and many other software is using the recognize elements (like searching for "faces" under images, and having facial recognition work to display only results with faces). Not too long ago one would likely believe a computer could not possibly recognize a human face, or much of the real world at all for that matter. But this is far from the truth, after breaking down how it works, and building things accordingly.

Originally Posted by Cavernio

I guess the biggest drawback to such a design is really making those secondary elements. What I'm describing, even if it saw a million cars, seems like it would only end up making a 'car' on it's own if it were exceedingly similar to the cars it's already seen. But maybe not, maybe it'd just be really, really fine balancing act of how weighted the likelihood of each element would be.

Perhaps hierarchy in a linear sense isn't the way to go, then. A generative, organic flow of recognition of what to draw, and context for the objects it's drawn, until something is detailed enough for the program's liking (in it's infancy, for example, grass would be a solid colour. Later it might add in some darker pixel variation. Eventually, making it draw grass over and over again, it might be able to replicate things nearly photorealistically, after analyzing photos of it in many ways).

From primary, secondary, to tertiary elements, I think it is restricting the potential of a program like this. But you have a great point in noting that there is a breakdown you'll need in order to make a drawing or painting look like a classically seen one; there is first a subject chosen, then a background drawn, then the characters shaped, detailed more and more, then something interacting with them.

Bit of a rambled response there~


One more thing, though; a big thing I'm pondering is how the creative ideas could come about. What conceptual processes are going on before something is even drawn? Would the program think on the screen, so to speak, by just drawing something, criticizing it, and then fixing it, and then continuing on? Or should the whole thing be decided first abstractly, as a logical cloud, an idea that's working as the mind of the artist, before the hands of the artist even begin to work. Perhaps it's a matter of having both conceptual and trying things out in the mix, which would make sense.

A program comes to the conclusion that it would like to draw an "apple" with a "screwdriver" "through" it. Such a string would be typed in by someone using the program, requesting something to be drawn based on words alone. These are abstract strings to the program, so it needs to associate with references. After it has a grounding in what these strings are referring to, it needs to think about what look defines an apple the strongest; which reference image represents apple the best it possibly can, and what can the program to to take those qualities to the next level. Afterwards it needs to think about the composition, how the apple is going to interact with the other elements, which it's just finished finalizing references from, the screwdriver. It breaks down the input strings and realizes that "through" means that the screwdriver is going to have to be passing the apple element in some way.

After the program thinks for awhile, working very hard to understand what is expected, it can then begin to work with trial and self-critical error to try to construct the drawing. After it feels it has completed the task, the user can input a suggestion string. "It needs a background. Make the apple blue. Make the apple drip with juice where it's penetrated". As well, the user can choose an option to "evaluate" and fill out a form, criticizing the drawing. The user can then shape the kinds of changes they would prefer the drawing to have.




I'm getting ahead of myself, I know, but what do you think about how a program would conceptualize abstractly and then apply the concept visually?

EDIT: @UNGH, thanks for the recommendation~ I'm already reading up about it in a way, but I looked into that book aswell. Can't wait to dig in~ I'm currently reading "Godel, Escher, Bach" which talks a LOT about the essence of artificial intelligence and same with "I Am A Strange Loop" which is basically an epilogue of G.E.B. Really interesting take on how feedback loops can become generative and begin to make decisions based on being a theorem or nontheorem of an axiomatic principle (such as: -P--Q--, as xPyQz, x * y = z and it explains how through certain adaptations you can sort odd and even numbers of dashes to compute certain tasks. --P--Q---- is a theorem and --P-Q--- is a non-theorem). I'm like only slightly on the brink of understanding the scope of that book though, fuck @_@