If you have died with Christ to the elementary principles of the world, why, as if you were living in the world, do you submit yourself to decrees, such as, “Do not handle, do not taste, do not touch!” (which all refer to things destined to perish with use)—in accordance with the commandments and teachings of men?
— Colossians 2:20-22
When we compare Peter’s testimony in Acts 3 with Paul’s meeting with James in Acts 21, it’s clear that the church in Jerusalem had changed. Perhaps the majority there felt that only those of Jesus’ family should rule the church. After James the apostle (not the brother of the Lord) was martyred, many of those who had a strong testimony, like Philip, left Jerusalem and spread the gospel elsewhere, leaving the church somewhat open for the religious ones to rise up.
History tells us that the situation in Jerusalem eventually became very chaotic, because they insisted upon keeping the old religious traditions. This is why Paul warned the saints in Colossae to beware of the elements of the world and the traditions of men, even though they may be something of the top religion ordained by God.