Modelling creativity

[Spenner]

I've been thinking of what might be involved for a computer program that reveals a form of creativity, given that it has tools to analyze things around it and constantly re-associate with previously learned knowledge.

If one were to program in some fundamental rules, such as composition, shape and contrast, understanding focal points and how one might be interpreting the elements when seeing it (to a computer, generated static is definitely an interesting picture to analyze; however to a human, we will not spend time picking up patterns of how such small pixel patterns might have been generated). A computer model like that would need to understand broader, more specific, modular elements that do not have to do with the fundamental processes that generate the pixels on a screen. It needs to look at the bigger relationships at play; how does shape A work with shape B in the context of what's on the screen? What theme can be illustrated?

However I think the bigger problem would be to try to convey to a computer exactly what things such as a "theme" and a "pretty picture" might be. Logically, it could create something which would be the bare minimum of what might be "compositionally correct" or the bare minimum for what a "colour-cohesive image" might be. A challenge would be to not let it stop at once analysis, but to constantly analyze what it's done previously, to find mistakes, to build more relationships with the other shapes it can compute.

What would those relationships be? On a very fine level it might be something as simple as "this grouping of pixels must not have a dense packing of pixels within a 100 pixel radius of the surrounding shapes", which could be condensed into a modular packet called "spacing". To get a computer to not choose the value of spacing needed for each drawing randomly might be tricky, because it needs an influence and reason for choosing that value; this could only be established by having the program analyze a large number of artworks and trying to compute why certain things work in each drawings, and projecting those ideas on the screen.

All this being said, it's not exactly creativity as much as it is a logical, straightforward approach to generating art. The trick would be to break the logical barriers and to influence trying things that are outside of the box, by having possibility generated from the imagery it's seen. A creative outcome of having it analyze paintings and finding out what aesthetics have been employed for each, would be to then compute a painting on screen that is an "average" of all techniques used. Something pretty abstract that would be hard for a human to decipher, because our individual minds influence what's good about a drawing, there is no objective reason for a colour choice being good for most people. To a computer, if it were to decide that, since a lot of paintings it's looked at were warm in colour tone, deep in contrast, and sharp in edges of defined shapes, that such characteristic looks are what makes a good painting (trusting that we show it examples of what collectively are considered very good pieces), it should be able to project those ideas in an original piece.

A missing point in all that would be choosing what to illustrate conceptually. A very technical projection of these rules might as well be something jumbled, messy and distorted, that has to significance to the human eye, and is unrecognizable. A computer model's "decided creativity" and a human's idea of creativity conflict dangerously because of what appears to the computer as merely an abstract view of something we humans see very clearly defined. It would be very important to have a strong grounding in shape recognition and being able to relate those shapes to an inventory of human and english concepts. Having an english dictionary plugged in, being able to respond back in a human-like fashion, would be key. That sounds like it might just need a rich database full of examples. Unless we can find some sort of algorithm for thinking and seeing like a human does (which frankly is a very dummed down version of the potential of what a computer might be able to observe given the right tools-- imagine programming a computer program that is as good at math as a first grader, and as good at english as one, instead of being perfectly instinctively clear with every computation). Being able to fill in the gaps, ask those extra questions that come out of abstractions, might be what separates a calculator from a more organic, human-like minded calculation. A robot that has to draw out it's work, from step to step, occasionally misinterpreting things because instead of an inherent value of 1 or 2, it is instead working on the associated shapes of those numbers, and the context it has on the paper, it's surrounding shapes, and many other countless variables.

I suppose I just wanted to rant about something I was thinking about, but I'd like to know what other people think about the potential of having any form of generated creativity. I'm far from implying that I've covered all of the necessary grounds here (I haven't even touched it to the slightest degree @_@) especially things like looping self-referential systems that are going on which are constantly becoming more complex, blahblah I won't let myself dig a deeper hole for my uneducated thinking. Just a curious rambling. What say you?


EDIT: I'll add on a few extra thoughts I was having aswell. I was on the back deck of the house I'm saying at, looking at a small fluff ball floating around, but in between the pickets of the rail of the deck. I was automatically seeing how one space in between the pickets related to the next one, not in one continuous sequence, but in little packets. I'll make a quick lil image:



Now, in that example, the next portion of line isn't necessarily related to the previous one. However, there have been variables of physics in the surrounding environment that have been applied to the previous frame to get the feather's path to shift to how it is illustrated in the next.



In this example, it's using the first frame (1) and constructing an idea based on given shape, but also based on a bit of a random influence of chance. Say that it has a set of rules that prevent the lines of the drawing from extending past more than an inch more than the original diameter of the circle (for example, just some kind of aesthetic rule).

A random decision to draw more lines off the drawing in (2) will illustrate no more relation to (1) than that simple rule. However, in (3), it has assessed the difference in characteristics between (1) and (2) and seen that the straightness of the corner in one of the little legs is out of character with the curvature of the rest of the drawing, making it out of context. It will then solve the context by erasing that limb and giving it curvature which is cohesive with the rest. The same thing happens in (4) and (5), though it's not really as accurate because after the first mistake in (2), generally, the rule should apply for the next set of drawn elements.

Just a very very small way that a computer could associate and build aesthetic cohesion with a very abstract "theme". The complexity of the drawing would be initially decided when envisioning the relative size of the ideal end image.

It wouldn't be an objective rule after computed for this drawing-- each decision will be individual from the last, with new parameters chosen given what's already been done. Perhaps a menacing face makes it's way onto this little crab in frame (6), making the limbs feel out of context, so in (7) the computer wishes to change all of the elbows of the crab's limbs to sharper corners. This wouldn't really be the case in my example though, because it's already been established that there is curvature in the general theme, so a happy face (with another element of curvature in the mouth) would feel far more cohesive.