It is only as religious of an issue as murder or stealing or any other violation of rights crime. Either it is a violation of rights or it isn't. And you can determine that with out resorting to religion, by looking at two issues:
1) Is there a human or not. This is a scientific issue. All biology text books agree that a fetus is a unique, living human...with a unique set of DNA defining a unique human different from the mother. In the past, those who support the right to abort have argued that a fetus is not human, but we see that argument falling away because it is becoming harder and harder to maintain that argument in the face of science. Religion has nothing to do with this determination. (The people who still argue this are usually not up to date on the logic of this issue. The real crux of the debate is now the second point below...what do you do in the event of a conflict of rights between humans.)
2) Give than that there are two humans involved (mother and unborn child), which rights prevail in a conflict between the mother's right to freely act in any way she wants and the unborn child's right to live. Granted, many religions state that all humans have a right to life, and that resolves the issue for many, but you don't have to resort to religion to decide this issue. Atheists are perfectly able to analyze this issue, and decide that the right to life trumps a right to freedom of action, especially when the one which possesses the right to life had no say in being placed where he or she was placed. The unborn is 100% innocent in creating the situation. [Now, in cases of rape, the mother is obviously 100% innocent as well, and we can argue about those rare cases. I happen to think the child still wins, but we can have a reasonable argument about that special case if you want. But, there is no use arguing those rare cases if we can't agree that a 100% innocent child's right to life trumps the right of the mother to be rid of the child she created. I would say that if there is to be a discussion, we should argue the main issue first, since it sets the groundwork for any exceptions.]
Again, this is not a religious issue any more than any other rights violation crime, such as murder or stealing. This is not a crime which has no human victim, like taking the Lord's Name in vain...a "non-consequential," purely religious issue in which there is no human victim, and which the law should not address. The secular law is perfectly capable to properly address "consequential" issues of humans being victimized without resorting to religion. Per item one above, there is a human victim, so the secular law can and should resolve a conflict of rights between two humans.
Granted, many people don't like how this issue is naturally resolved, so they say that it is a religious issue, or a personal decision...but it is not a personal decision any more than it is a personal decision by one person to violate another person's rights. If I open my front door Saturday morning, and discover a baby in a basket on my front porch, I don't get to throw the baby in the trashcan because it violated my property rights, or because it has no right to demand that I feed it or to ask others to do so. The right to life trumps my right to not be bothered--not a religious issue.
You can argue that only a subset of humans actually have human rights...but then a right is not really a human right if not all humans have it. So, the right to life ceases to be a human right. It is just something that people have if they happen to live under a legal authority which gives them that right. It is not a right, it is a government benefit. Most people, even atheists, don't like that outcome.