In a previous publication by the same authors, a computational design system was described that identifies aesthetical color combinations. In that work the type of color aesthetics pursued was to be determined prior to the design representing a given design objective. Complementing the previous study, in this work, the type of color aesthetics that is most suitable for a given scene at hand is pursued taking into account the color and geometry of an existing situation. This is accomplished by bringing the parameter that characterizes the type of aesthetics into computational design, i.e. treating it as one of the components of the decision variable vector subject to identification by multi-objective evolutionary search. The parameter's influence on aesthetics is investigated theoretically, as well as by means of computer experiments. The contribution of the study to Architecture is provision of a firm base for some common architectural knowledge as to the color aesthetics of buildings. In particular light is shed on the aesthetical dependence of a building's color to the color of its environment.