You feel guilty because you are a moral being with a conscience. Our conscience makes us aware of our moral and ethical responsibilities towards God and one another. A well taught conscience is a gift from God and will alert us to what is wrong and condemn us when we do it.

Guilt makes you hide away from God and those who speak about Him. It may cause you to dislike ministers of the Gospel because they remind you of God’s holiness and make you feel guilty.
Many harden their consciences so that they become insensitive to feelings of guilt. If you struggle to suppress guilt you could upset your physical and mental health.

A guilty conscience can drive us to Christ like a pain drives us to the doctor. When we feel guilty before God we see our need for a Saviour. He is the doctor of our souls. The woman in the quotation above was a prostitute. She went to Jesus for moral cleansing. We all need cleansing from the guilt of sin. A tender conscience is a good thing if it brings us to Christ.

A bad or evil conscience can immediately be relieved when we confess our sins to God and trust Christ as our Saviour. The guilt is cancelled immediately like a great debt. The sense of relief is instantaneous. The cross of Jesus is God’s way of dealing with our sin. If we confess and forsake sin, God will forgive us.