Barplot function in R

Much of my backtesting platform is text driven. Not that I'm opposed to graphs...just felt my time was better spent developing the foundation for the platform before adding bells and whistles. Little did I realize how difficult it is to find a simple graphing engine for the platform. Problem is...I'm old school...couldn't care less about flash graphs. Keep it simple.

Since I'm using python...figured I had to give the matplotlib library a try. It is nice...simple...but something was missing. Couldn't put my finger on it. So, dug around and played with the R language plotting libraries. A bit more my speed...though a bit particular in the settings. Anyway, here's a function I wrote to generate bar charts using R with a replacement for pie charts in mind...


#-----------------------------------------------------------------
# Simple bar chart - use instead of pie chart when possible.
#-----------------------------------------------------------------
barPie <- function(xSeries, chTitle="Your Bar Chart", xLab="X Label",
xDesc="%")
{
xSeries <- sort(xSeries)

# save off original settings in order to reset on exit
oldPar <- par(no.readonly=TRUE)

plot.new()

# set page margins in inches
par(mai=c(1,1.5,1,1))


# pad 30% for labels
# start plotting at 0.0 unless negative
if (min(xSeries) < 0.0)
{
xLim = c((min(xSeries) * 1.3), (max(xSeries) * 1.3))
}
else
{
xLim = c(0.00, (max(xSeries) * 1.3))
}

# horizontal barplot in color baby!
bp <- barplot(xSeries, horiz=T,
xlab=xLab, las=1, col=rainbow(length(xSeries)),
xlim=xLim,
axes=F, cex.names=0.7, main=chTitle)

# if x negative then start label at 0.0
# otherwise, start label at value of x.
xVals = ifelse(xSeries < 0.0, 0.0, xSeries)
text(xVals, bp, paste(xSeries, xDesc, sep=""),pos=4, cex=0.65)

# format x axis
xRange <- formatC(pretty(xSeries), 1, format="f")

axis(1, at=xRange, labels=as.character(xRange), cex.axis=0.75)
box()

#restore par value to previous state
on.exit(par(oldPar))
}


Used data from my portfolio to plot sector allocations and called the function...

sectors <- c(10.64,119.83,162.66,66.48,71.78,35.44,32.77,161.17,53.91,
101.81,53.38,231.45,31.24,103.01)
sectors <- round((sectors/sum(sectors)*100.00), 1)

# write to png driver
png("c:/taylortrade/rlang/sectors_test.png")

barPie(sectors, "Sector Allocation", "Pct Allocated")

# stop writing to png driver
dev.off()


And here's the result...

What I'm Researching...

Visualize real-time data streams with Gnuplot Posted: 22 Sep 2008 11:26 PM CDT perl script feeding Gnuplot to plot streaming data in real-time. Interesting find...hat tip to Hackszine.com

The plotrix Package (pdf) Posted: 22 Sep 2008 10:21 PM CDT need to review this...looks like an all encompassing plotting package for the R language. Barcharts, piecharts, gantt charts, oh my!

What I'm Researching...

Writing R Extensions (pdf) Posted: 21 Sep 2008 09:28 PM CDT the how-to of creating extensions to the R language.

The Hazel Tree - A Python Compendium Posted: 21 Sep 2008 07:21 PM CDT central location of texts covering python.

The fTrading Package (pdf) Posted: 21 Sep 2008 06:25 AM CDT R language package containing functions for ohlc plots, sharpe ratio, sterling ratio, maxDrawDown, rolling analysis, ema, sma, emwa, etc.

What I'm Researching...

Quick-R: Home Page Posted: 21 Sep 2008 01:37 AM CDT nice site covering all the R essentials - reading data, graphing, stats.

R mailing lists archive Posted: 21 Sep 2008 01:28 AM CDT search the R language mailing list.

Rtips Posted: 21 Sep 2008 01:26 AM CDT cookbook of the R language

Producing Simple Graphs with R Posted: 20 Sep 2008 10:47 PM CDT examples of various charts (line, bar, pie, dot, histogram) in R language

Creating Charts and Graphs with GNU R Posted: 20 Sep 2008 10:45 PM CDT brief tutorial covering pie, bar, scatter, and line charts in R language.

R Graph Gallery :: List of graphs Posted: 20 Sep 2008 10:26 PM CDT great gallery of R Language graphs.

R Graphical Manual Posted: 20 Sep 2008 10:16 PM CDT gallery of R language graphics by category such as Finance, Econometrics, SocialSciences, etc.

R Graphics Posted: 20 Sep 2008 10:13 PM CDT book on the graphics available with the R language. Very nice!

Chart of R Colors Posted: 20 Sep 2008 10:12 PM CDT table of colors available in R language.

Using Color in R Language (pdf) Posted: 20 Sep 2008 10:10 PM CDT showcases all the colors available in graphing with the R language.

Getting Started With Matplotlib's OO Class Library Posted: 20 Sep 2008 08:34 PM CDT brief summary of matplotlib's api.

Using Python for Interactive Data Analysis Posted: 20 Sep 2008 08:26 PM CDT Several excellent examples of using numpy and pylab (matplotlib) together.

scienceoss.com » Blog Archive » Creating a custom bar plot in matplotlib Posted: 20 Sep 2008 03:33 PM CDT interesting code examples of bar charts in pylab.

What I'm Researching...

Python: package matplotlib Posted: 18 Sep 2008 11:20 PM CDT api reference for matplotlib - not a cookbook of examples - just command and descriptions. Use as reference guide.

Plotting with Pylab - python Posted: 18 Sep 2008 11:13 PM CDT drop-dead example of plotting with pylab versus matplotlib's api. would like to use pylab's api...if at all possible.

Sofeng's Blog: Simplistic Python Thread example Posted: 18 Sep 2008 11:02 PM CDT drop-dead example of threading in python.

Sofeng's Blog: How to use *args and **kwargs in Python Posted: 18 Sep 2008 10:59 PM CDT drop-dead simple explanation of variable length argument lists in python.

Sofeng's Blog: Emacs 22 Command Index (sorted by Category) Posted: 18 Sep 2008 10:58 PM CDT breakdown of emacs commands - not sure if format is the best...but worth saving for future.

Sofeng's Blog: Example pie charts using python and matplotlib Posted: 18 Sep 2008 10:38 PM CDT really like the look of these pie charts. use for sector allocation page.

Sofeng's Blog: How to draw a simple line using python and the matplotlib API - Posted: 18 Sep 2008 10:34 PM CDT drop dead simple example of using matplotlib.

Goldblog - Python - Creating Bar Graphs with Matplotlib - Corey Goldberg Posted: 18 Sep 2008 10:20 PM CDT example of bar graph in python. like the format of the graph.

Vim is a beautiful tool Posted: 18 Sep 2008 01:04 PM CDT nice summary of the top shortcuts used in vim.

Portfolio Performance for August 2008

Some very nice weather happening here in Missouri. In fact, in all my life I've never had weather like this in August/September timeframe.

Back in Texas, it's usually some of the hottest times of the year. But, feels like Fall already in Mid-Mo.

Can't wait to see the leaves turn colors. Of course, my kids are looking forward to piles of leaves to jump and play
in.

Maybe this weather change will bring the market out of the doldrums. By the way, the picture above was taken while visiting the Devil's Ice Box.

And now for the August returns...






Still, nothing going on with the market or the trading system. Though, plenty to do with the system testing platform. I'm exploring the numpy library for python to determine how big a role this library should play in the platform's architecture. For those curious...check out my delicious bookmarks on numpy.


Later Trades,

MT

Sector Allocations for August 2008

Quick look at the sector allocations for the portfolio. Please note that cash is not included in the breakdown. Next on my development list.

MT

Portfolio Performance for July 2008

July ended with a surgery for yours truly. I had my gallbladder removed. I had a rough few days of recovery but starting to feel normal again.

I had a set back this month in regard to the testing platform. The redesign has severely extended the simulation runtime. So, I've spent much of my time finding the bottlenecks of the code and determining the optimal solutions. Very thankful python has such great profiling tools at the ready.

You know, performance tuning code is sure a lesson in humility. All my years of programming...I feel I should have a good handle on the slow areas of my code. It only takes a profiling report to show how wrong I am.

Very similar to trading the market. Trading decisions are based on our experience. Where we think the money-making opportunities are. Only takes a simple backtest to showcase how wrong we are.


And with that...some charts for the month of July...





Not a whole lot of movement in the portfolio occurred the month of July. Just another month of sitting and waiting. Letting the system do what it does best...waiting for the market's next move.

Later trades,

MT

Portfolio Performance for June 2008

June was a great month for the family. My major project came to a successful close. And with that a huge amount of time freed up. With that, we made our first trip back to Texas in 2 years. It was a very nice visit. We were able to see family, play on the lake I grew up on, and enjoy some much needed Texas culture.

June wasn't as great for the portfolio. But, did manage to beat the market for the month by a fairly large margin putting us in the lead for the first time this year. I believe a big reason for this outperformance was due to the pulling of weeds back in March and April. This allowed new positions to take hold in the portfolio garden.

Goals for the month of July?

• Complete the upgrade that will enable testing of multiple systems against a common portfolio. This will allow exploration of various allocation methods. And judge the effects of merging long-term systems (trend capturing) with short-term systems (volatility harvesting).
• Wrap-up the upgrade that allows ranking of signals and existing positions in order to initiate trades.
• Modify how cash is tracked in the testing platform. Currently, cash is a static variable. Goal is to convert cash into a unit series to enable dynamic pricing. This will enable sweep accounts in the portfolios. And allow the switching of cash instruments.

Later Trades,

MT

Portfolio Performance for May 2008

I've been neck deep in one heck of a massive project at work. Today is the first day in months that I've got a breath. And with that breath, I figure I'd post the portfolio's performance from the past few months.



As you can see, the past two months have been the best months of the year for the portfolio. Still not great...but much better than before. In fact, in the chart below, you can see the portfolio's monthly return finally beat the market for the first time this year.



You might have noticed a change in the charts from the prior month's reporting. That's due to a change in Icarra's chart rendering methodology. This new methodology proved more difficult to post images to this blog. So, I rolled my own portfolio software. Which is definitely a work in progress. Over time, I hope to develop the capabilities to track and share more details on trade statistics from this software.

My ultimate goal is to merge my backtesting application with this portfolio software. This way I could compare simulated results with real-time performance.

What are my next investment experiments?

• I'm curious how the elements of permaculture can be used in selecting investment vehicles for the portfolio.
• Would Janet Brown's investment methodology improve my system results?
• When receiving a large number of buys for a given sector...is it better to buy as many stocks as I can under my risk limits? Or more appropriate to opt-out of all the individual stocks and buy an ETF in that sector to capture the obvious sector-based trend?
• Measuring systems that utilize different levels of cash throughout their system life cycle. If system A's performance is better at all conventional levels (Sharpe, ROI, etc.) than system B's...yet system B uses 50% less cash...are the comparisons valid? I don't believe so...but how do you compare then? Via converting unused cash to utilize market-based returns? This is a tough one.

That's it from here where we've had so much rain in Missouri that I'm beginning to think I'm back in East Texas.

Later Trades,

MT

Portfolio Performance for Feb 2008

Another month in drawdown. This is where the lesser experienced would begin to question the validity of their system. Has something stopped working? Did the market change? If you design your strategy based on a helter skelter of methodologies...you may have a point.



But, if you build your system based on capturing the market's inherent behavior: stocks can exceed 100% returns but never exceed 100% loss (unless on margin). Then you know when the market turns south...you'll turn south. Nothing you can do but weather the storm. The fortunate thing is you have allocated your money based on this knowledge...that there will be storms. And you must wait for the storm to pass...the skies to clear...and the markets to return.

Below are the sector breakdowns for the portfolio. The cash level increased 35% from last month's level. Again, the increase in cash will reduce the portfolio's drawdown should the market continue to stall. But, decrease our returns should the market recover.



In fact, you can see in the chart below the difference the cash level is making to the portfolio.



February's drawdown is beginning to slowdown in comparison to the benchmark. As we move further and further into the market's downturn...the portfolio will begin to break off from the benchmark due to the rising cash level.

This rising cash level is due to 29% of the portfolio's positions hitting trailing stops since Jan 1st. In that same timeframe, our new new positions increased by 4%. Is this good or bad? Neither. Just the way the market works.

On a side note...the weather here in Missouri is toying with us. You can tell spring is trying to push it's way in...but the winter isn't giving up without a fight. Crazy thing to be riding bikes in short-sleeves one day and bracing for snow the next. Markets and weather...what fun.

Later Trades,

MT

Portfolio Performance for Jan 2008

I would like to have done a better job at my first portfolio performance post. But, I'm afraid I've been extremely busy at my day job. So, please be patient. My hope for future performance posts is to include much more information about the returns and portfolio breakdown.



One of the difficult items I've found in preparing for posting of portfolio performance is the lack of good tools. I've tried quite a few products the past few months and was about to give up hope until I found Icarra. Sweet product. One of the hardest things to create is a simple product. So many developers miss the mark...the tendency is to focus on the bells and whistles. Icarra hits the mark with it's great charts, sector breakdowns, and ease of trade entry.

This is a good segway into developing a portfolio. As an investor, you have unlimited choices. Question to ask yourself...do you focus on the bells and whistles? Or do you hunker down and keep your choices simple...your strategy simple. If you find yourself changing your portfolio on every market downturn just because you feel like you should be doing something...then maybe it's time to refocus. Are you really improving your portfolio returns? Or just keeping yourself busy?

Now, back to portfolio performance. There are some things to note. I have not entered my actual entry prices into Icarra. Just too much data to sift thru in order to accomplish that. Instead, I have entered all the positions as if I bought them on the last day of 2007 (12/31/2007). From this point forward I will enter all sells, new buys, dividends, interest, etc. into Icarra. But, keep in mind...all performance is based on the clean slate of 2008.

Below is the breakdown of sectors in the portfolio. This is one area that I found some differences in the chart and the actual breakdown in Icarra. But, useful nonetheless. I'll contact Icarra to find the reason for the difference and post in the next performance release.



You can see, I now have a very large cash position in the portfolio. I don't like this, I have to admit. This will cause the portfolio to lag should the market gain back momentum. But, this is the normal process of the system. During market downturns like we've had...the trailing stops are hit and the bugs in the portfolio are killed. Then we have the waiting game for new signals. The goal, of course, is to be 100% invested in the market...that is what generates our returns. But, all seasons have their winters...and this is where we bundle up and wait for the ice to melt.

Another area I want to point out in the sector breakdown is the lack of consistency in the sector weightings. My goal is to have a better balance in the sector weightings. These sector weightings cannot always be controlled due to the nature of the system's signals. Some sectors may be very cold and other sectors can be very hot. Based on the weightings above, the services sector was pretty hot for 2007....and have held up relatively well in the portfolio during our bug killing in the recent market downturn.

Finally, I want to review the monthly returns of the portfolio...



Pretty large differential between the portfolio's monthly returns and the S&P 500's. The system is more volatile than the market and as result drawdowns and gains are larger. This is par for the course with trend following systems in general. It takes a lot of patience to trade a trend following system. Especially, a long-term trend following system. But, I truly believe, that's where the returns are in the long-term.

Can this be improved? Possibly. There are ideas I have yet to test that could trim the portfolio's volatility. And hopefully, I'll get a chance to test these ideas in the months to come.

If you have any questions or would like to see other aspects of the portfolio...drop me a line.

Later trades,

MT

This post is for information and entertainment purposes only. Under no circumstances does this information represent a recommendation to buy or sell securities or any other type of investment instruments.

Quote of the Week - 01/16/2008

The last spike of the transcontinental Railroad is driven at Promontory, Utah in 1869.

"Look at a stone cutter hammering away at his rock, perhaps a hundred times without as much as a crack showing in it. Yet at the hundred-and-first blow it will split in two, and I know it was not the last blow that did it, but all that had gone before." -- Jacob A. Riis

MT

Quote of the Week - 01/06/2008

"Risk is trying to control something you are powerless over." -- Eric Clapton

Hope your first week of the new year was a good one. I've spent a bit of time planning out the new year (talk about your risk). One of the first items on the agenda is a visit back to Texas to see the family. In addition, some changes will come to the blog.

My testing platform development is coming to an end and I should have more time to research and discuss portfolio ideas/strategies. So, each month I plan to review my personal portfolio composition: returns, drawdowns, sector breakdowns, you get the picture. The first few postings will be a bit raw...but hopefully over time they will improve as will the returns.

Later Trades,

MT

Quote of the Week - 01/01/2008

"I sometimes meet people who say, I'm going to be this and I'm going to be that. You feel kind of bad for them because they're limiting themselves. It's different from having an enthusiasm for something and seeing where life takes you. I feel lucky to never have planned to go into what I did. I always said, "All I want to do is make things, whether it's drawing or writing." If I'd said, "I'm going to be a director," it probably wouldn't have happened." -- Tim Burton

Tim's quote is a great one to start the new year. Focus on what you enjoy and let life handle the details.

Happy New Year!

MT

Quote of the Week - 12/26/2007

"Pick a mountain in life to climb and don't stop til' you reach the top. When you reach the top...be lookin' for the next mountain to climb." -- Life philosophy of Helen Taylor

My Mom died 6 years ago today. I lost track of the times I heard from her, "climb that mountain", when life turned difficult. And the many times I responded, "I'm tired of climbing."

Never would have thought her mountain mantra would have such an impact in my life. That recalling those nagging words would continue to push me forward in life. Even after she's gone.

Still an inspiration, after all these years.

Thanks, Mom.

Mike

Merry Christmas - 2007

The picture above is from December 2006. Our first snow in Missouri. There is a bit of snow on the ground today...but quickly melting. We may have more snow tomorrow.

Hope everyone is having a great Christmas. I've spent my time drinking coffee my daughter gave me and putting together all the presents for the kids. Next step is grilling some nice tenderloins wrapped in bacon. No thoughts about the markets or programming. Just kicking back and enjoying the day.

Oh, and very thankful for the great gift a friend of mine gave me last year. I've never owned a cordless drill before. A friend of mine felt sorry for me and gave me a full set of Dewalt cordless power tools. All these years I've been cussing at putting toys together...until now. Wow, what a difference Dewalt makes. Thanks, buddy!

MT

Quote of the Week - 12/17/2007

"Do what you gotta do so you can do what you want to do"
-- Denzel Washington

MT

To Design or Code?

"The one who does the work decides." -- KDE principle

Jeff Atwood over at the Coding Horror blog discusses a fascinating problem in software development. Doers and talkers. Designers and coders.

I believe all developers need to have a bit of both in their toolbox. Mainly, because the first design is most always changed due to scope creep (failure to see all the pieces to the puzzle). If you spend all your time talking about that first design...you never get to coding. And if you can't get to the coding...you'll fail to find those missing puzzle pieces. And fail to deliver a prototype for the customers to evaluate.

Designers, this means getting your hands dirty in order to create something to improve. Developing systems is an iterative process. Design, code, design, code. And yes, even code, design, code, design. The ultimate goal is to refine the process until you and your customers are satisfied. Whatever it takes. And yes, that means moving to the coding stage even when the optimal design has yet to reveal itself. It's really a Kaizen process. Small accomplished improvements to the initial design pays dividends to all.

Coders, this means before plunging forward hacking away at the problem...ask for feedback of your idea and possible alternatives to the problem. It's important to starting coding down the right path. One that encompasses as much information of the problem as possible. This means you'll need to bring some information to the design table yourself. Perhaps a bit of discovery coding must take place to figure out what elements are involved and possible problems or bottlenecks in your proposed solution. This also helps in keeping the design discussion focused.

So, how does this apply to investing? Well, how many investors/traders do you know that invest without a plan? Without a design? An overriding investment philosophy? Just plunge ahead into the market?

These type of investors would be well served by stepping back a bit and design their investment model. Then ask for feedback of their proposed design. It's okay to perform some discovery trading first. Determining how the market handles your ideas. But, gather what you need and then design. Then invest with the goal of continually refining your design.

What about investors/traders who are afraid of the market? Have not found an investment model that is perfect? And refuse to step a toe in the market waters until they feel 100% comfortable in their design? Problem with this thinking is knowledge requires experience. And nothing is ever 100%...especially in the market. So, create your investment manifesto and then try it out. You can't improve upon something that isn't there to improve. And you can't design a successful investment strategy if you don't have market experience.

Later trades,

MT

Breakthrough Programming

“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.” -- George Bernard Shaw

Scott over at the scottberkun.com blog links to an amazing paper on Managing for Breakthroughs in Productivity. The article discusses how breakthroughs occur...and don't occur. My favorite quote...

For breakthroughs to occur, people must be given a chance to do work than can not be proven: ambition and risk are necessary for breakthroughs. If individuals are not trusted to take risks, breakthroughs are unlikely.

Spot on! Programming and risk goes hand in hand. As do programming and ambition.

To be an effective programmer you must have the ambition to automate tasks...all tasks. That's your job. If you spend your time manually putting widgets together...then you're not really programming.

And in order to automate tasks...a programmer has to take risks. Cause you are programming something that has never been done by a computer before. At least that computer. And most likely, never been programmed by you before.

MT