Opening borders is like removing the doors from your home
When we put locks on our doors and fences around our home, we do so with the intention of protecting what we have created. A country has a culture that supports what it created, and borders help to protect that.
Wealth and culture are not the only things worth protecting by controlling borders. Any human investment that has resulted in some tangible creation that is then desired by other humans (or non-humans) will need to be protected. This results from base drives that affect behaviors. Some of these drives are directly related to physical survival, hunger, thirst, cold, etc. and others are emotional, greed, lust, laziness, etc. For a stable society to exist, the physical needs must be met, and the anti-social emotional drives need to be managed. Culture is one factor that contributes to this control. Wealth and culture are both examples of tangible creations that require an investment to create, and therefore induce a desire to protect.
Analogizing one's home helps to make the argument for borders more tangible. We all understand that our homes are places that are created, and require continued investment to maintain, both emotional and physical. We all know either from direct or indirect experience that not all people will respect the investment you have made, and given the opportunity, will take it, thereby enjoying the benefit of your investment without the effort of creating it themselves. So we make a law, and invest in a police force to enforce the law, and execute consequences for breaking the law, all in order to discourage people from following our innate human drive to maximize gain while minimizing effort, which is at least partially driven by greed, lust, laziness, etc. The severity of the consequences is driven, at least in part, by what the culture, rather than individuals, values most. The fences and the doors are reminders that there is a law, and there are consequences for breaking the law. Our homes have something of value that we want to protect; in the same way, a country has borders because it has something it wants to protect.
Borders are incredibly important to separating regions that are culturally and politically different from each other. They "reflect the vagaries and irrationalities of history." 
When borders are too quickly changed or even dissolved, it can lead to political unrest due to cultural, historical, and ideological differences among different groups of people, such as in the Balkans in the last 150 years.
Further, borders reflect the innate human desire to own and protect property. 
It's similar to the idea of fences clearly delineating one person's house and property from that of their neighbor's. A clear boundary marks the distinct and separate entity of two places and also establishes where a person can and cannot tread. 
Without borders, the human desire to own and gain property would be out of control, thus leading to conflict. By establishing boundary lines between two different places, friendships and alliances can be built instead because the borders acknowledge and respect individual regions.
Reflecting on borders that were poorly placed in the past can also shed a light on possible effects of having no borders. In Africa, European colonists set borders that failed to "take into account the vibrant ethnic and cultural diversity of the continent." 
As a result, skirmishes and civil wars have broken out in various parts of the continent. Without borders at all, this competition between different groups of people would only intensify.
Opening up borders doesn't necessarily mean that the ideas of owning property and territory are abolished. Countries can still issue passports and maintain sovereignty. However, immigrants would be automatically accepted at all standard ports of entry. 
The influx of immigrants would boost the economy. Although poorer countries would see an exodus of emigrants, this may actually prove beneficial as the dwindling labor pool would keep wages high. In fact, standard estimates have found that open borders would ultimately double humanity's wealth production.
The presence of borders is also a perpetuator of violence. Modern immigration rules are stringent in order to keep people out. Intensification of border enforcement has led people to take more perilous journeys in order to cross and has led to thousands upon thousands of people dying each and every year in an attempt to cross borders. 
Violence of this nature would be eliminated with the removal of borders.
[P1] Open borders means giving up sovereignty over one's territory.
[P2] Violence would increase without the presence of borders to keep people in line.
[Rejecting P1] Open borders doesn't necessarily mean abolishing sovereignty.
[Rejecting P2] Violence would decrease because people would not be forced to take perilous risks in order to cross borders.