From my personal experience and from my observations of those around me, I think our physical environment affects us in some important ways. Whether it is the natural characteristics, urban planning, architecture, or interior design, many aspects of our lives and our emotions are influenced.
The natural characteristics, such as weather, sea level, beaches, oceans, mountains, rivers, and lakes, both facilitate and limit certain lifestyles and hobbies. They determine how the average person might dress and what they might have to deal with on a daily basis (shoveling snow, preventing heatstroke, staying dry in the rain, keeping warm, etc.). Most importantly, they affect people's moods in one way or another - sometimes quite drastically.
Urban planning is another factor of our environments that has a tremendous impact on our daily lives. The average person's lifestyle is almost predictable based on how their area is designed. For someone like me, the car-dependent nature of most North American cities is depressing, inconvenient, and just generally unpleasant. The lack of accessible greenspaces reduce the habitants' wellbeing. The suburban design created the "soccer mom" lifestyle, taking away both the independence of children and the other aspects of the mother's life (career, social life, self-care). Even economics and air quality are heavily dictated by urban planning.
On a smaller scale, architecture is similar to urban planning; it is about being functional and beautiful. It is what gives an area its identity. We think of places by the cityscape, the iconic landmarks, and the general aesthetics of the architecture. These buildings and imageries make us feel something. They interact with us on a daily basis. People don't want to be in depressing buildings or be surrounded by them. People don't want to deal with problems caused by them (like how skyscrapers make the wind worse for pedestrians or how cold it can get living in an uninsulated/poorly insulated building).
Interior design is perhaps the most important to the average person, as it is the aspect of our environments that we interact with the most, whether that is where we work, where we sleep, or where we hang out. The proper lighting and decors make a space much more livable and enjoyable to be in. Intelligent design allows spaces to be functional and productive. Thoughtful details makes places feel personal and unique. People want to be in places that is the "right vibe" for them or for the activity they are doing.
As important as our physical environments are, it doesn't seem to me that people consider it very seriously nor very often. It feels like people would rather complain about various aspects of their environments than to do something about it or to move somewhere that is actually compatible.